{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building a Simple Unit Test for the Flogo Inference Activity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>The goal of this notebook is to build a simple CNN model to be used as</p>\n",
    "<ul>\n",
    "<li>A small tensorflow model to use as unit test for the Flogo Inference Activity</li>\n",
    "<li>A demonstration of building and exporting a model into a format that can be used in Flogo's inference activity.</li>\n",
    "</ul>\n",
    "<p>Therefore the \"problem\" we are going to solve with CNN is this:</p>\n",
    "<blockquote><p>Given an array of size \"size\" containing a gaussian determine whether the maximum value is in the lower half of the array (value=0) or the upper half (value=1).</p></blockquote>\n",
    "\n",
    "<p>Since this is a sample problem I will generate my own test/train data sets.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Notebook setup\n",
    "<ul>\n",
    "<li>Modules used</li>\n",
    "<li>Meta-parameters used within notebook.</li>\n",
    "<li>Basic functions defined to expidite coding</li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import Required modules - Numpy , Tensorflow, and Matplotlib/Seaborn with a few default packages as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline  \n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn\n",
    "import random\n",
    "import math\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Meta Parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "size=10               # Size of the Array to contain the gaussians - MUST BE EVEN\n",
    "\n",
    "lr=5e-4               #Learning rate of the CNN \n",
    "numfilters=15          #Number of convolution windows/fliters to use \n",
    "\n",
    "batchsize=50          #Training Batch size\n",
    "eval_size=500         #Evaluation set size\n",
    "\n",
    "pstep=25             #Number of training sets at which to print out loss, accuracy, etc. of training step\n",
    "estep=100            #Number of training sets at which to test and print the reults of the Evaluation set\n",
    "train_steps=2001     #Total number of training sets - Useful to have the one at the end to make the last set \n",
    "                     #   print out\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Defining useful functions\n",
    "<ul>\n",
    "    <li> \"bell_fn\"- A function that given x,x0, alpha and beta gives the resulting gaussian of: beta*exp((x-x0)^2/alpha)</li>\n",
    "    <li> \"data_point\" - Given the size of the array needed generate an input and the corresponding training label</li>\n",
    "    <li> \"create_batch\" - Given batch size create a batch of inputs/outputs that can be used for training or testing</li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bell_fn(x,x0,alpha,beta): #beta*exp((x-x0)^2/alpha)\n",
    "    xxo=x-x0\n",
    "    return beta*math.exp(-xxo*xxo/alpha)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def data_point(size=10,alpha=5,beta=10):\n",
    "    #adding some noise to the default alpha and beta\n",
    "    if alpha==5 and beta==10:\n",
    "        alpha=alpha+(random.random()*2-1)\n",
    "        beta=beta+(random.random()*2-1)\n",
    "    out = np.zeros(size)\n",
    "    loc=random.random()*size\n",
    "    out=np.array([bell_fn(i,loc,alpha,beta) for i in range(size)])\n",
    "    #print(loc)\n",
    "    return out,0 if np.argmax(out) <size//2 else 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_batch(batchsize):\n",
    "    x=[]\n",
    "    y=[]\n",
    "    for i in range(batchsize):\n",
    "        xi,yi=data_point(size)\n",
    "        xi=[[[a]] for a in xi]\n",
    "        x.append(xi)\n",
    "        y.append(yi)\n",
    "    x=np.array(x)\n",
    "    y=np.array(y)\n",
    "    return x,y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building the Graph for the model\n",
    "<p>As mentioned in the introduction, we are going to build a CNN to operate on the gaussian array of size \"size\".  We are going to use a fairly basic convolution/pooling/fully connected layered setup.  Because of the simpleness of the problem statement (and to make a smale model for the unit test) we will only use a single conv/pool/fc layer.</p>\n",
    "<p>It is worth noting that the conv2d tensorflow function is used with the second dimension and channel ranks set to 1 instead of using tf.layers.conv1d because conv1d has a bug that has been fixed in tensorflow 1.11, but currently conda only has tensoflow at 1.10.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Defining placeholders (Nodes to accept input) for the input gaussian array (X) and the training labels (Y).\n",
    "X = tf.placeholder(tf.float32,[None,size,1,1],name='X')  #[batchsize,size,1]\n",
    "Y = tf.placeholder(tf.int32,[None],name='Y')     #[batchsize]\n",
    "\n",
    "#My conv/max pooling/ fully connected layers.  Notice my Pooling is choosen to divide the data into a top and \n",
    "#   bottom half to make the identification of whether the max is in the top or bottom half easier.\n",
    "conv1 = tf.layers.conv2d(inputs=X,filters=numfilters,kernel_size=[2,1],strides=1,padding='same',activation=tf.nn.relu,name='conv')\n",
    "pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=size//2,strides=size//2,padding='same')\n",
    "pool1 = tf.reshape(pool1,[-1,numfilters*2],name='pool')\n",
    "logits = tf.layers.dense(inputs = pool1, units=2, activation=tf.nn.tanh,name='fcon1')\n",
    "\n",
    "#Using Softmax to get probabilities from the logits\n",
    "probabilities=tf.nn.softmax(logits,name='softmax')\n",
    "\n",
    "#Getting the loss, prediction, and accuracy\n",
    "loss = tf.losses.sparse_softmax_cross_entropy(labels=Y,logits=logits)#,reduction=tf.losses.Reduction.NONE)\n",
    "pred = tf.argmax(input=probabilities, axis=1,name=\"pred\")\n",
    "accuracy = tf.metrics.accuracy(labels=Y, predictions=pred)\n",
    "\n",
    "#Defining the optimization step\n",
    "train_step=tf.train.AdamOptimizer(lr).minimize(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating and initializing the Tensorflow session\n",
    "<p>Remember with TF you need to define the graph (above) and then create a session to \"run\" the graph.  Here we create the session and initialize the global and local variables in the graph.  Then because the trainable variables are pretty small here we take a look at the variables in the model.  Yup should be small enough for a unit test to be stored on github.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess=tf.Session()\n",
    "sess.run(tf.global_variables_initializer())\n",
    "sess.run(tf.local_variables_initializer())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variable:  conv/kernel:0\n",
      "Shape:  (2, 1, 1, 15)\n",
      "[[[[ 0.3124455   0.03911668 -0.22816731  0.18111172 -0.32070804\n",
      "     0.06484324  0.15507439 -0.2589414   0.07914296  0.23594603\n",
      "    -0.1644884   0.2390627  -0.4072155  -0.07656926 -0.29182953]]]\n",
      "\n",
      "\n",
      " [[[-0.3685084  -0.18050045  0.0896301   0.14999911  0.1548188\n",
      "     0.33474723  0.23378822  0.38202962  0.01755071  0.30117908\n",
      "     0.37522277  0.30340967 -0.34644172  0.2127724  -0.26057598]]]]\n",
      "Variable:  conv/bias:0\n",
      "Shape:  (15,)\n",
      "[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      "Variable:  fcon1/kernel:0\n",
      "Shape:  (30, 2)\n",
      "[[ 0.2446551   0.23777756]\n",
      " [ 0.4295577   0.2667931 ]\n",
      " [-0.42782715  0.05270955]\n",
      " [ 0.29815903 -0.11822918]\n",
      " [ 0.00050452 -0.42741573]\n",
      " [ 0.4272459   0.22994438]\n",
      " [-0.3497372  -0.25100434]\n",
      " [ 0.12461451  0.07820824]\n",
      " [-0.36327308 -0.11236423]\n",
      " [ 0.26886454 -0.2924248 ]\n",
      " [-0.28388584 -0.2989422 ]\n",
      " [-0.05223545 -0.2276599 ]\n",
      " [-0.18197387  0.40160194]\n",
      " [ 0.20518109  0.32966128]\n",
      " [ 0.1974729   0.3220531 ]\n",
      " [ 0.16404316 -0.3330704 ]\n",
      " [ 0.08136466  0.14260796]\n",
      " [ 0.38732037  0.17442259]\n",
      " [ 0.1788185  -0.38673717]\n",
      " [-0.22687209  0.0406419 ]\n",
      " [ 0.24908563  0.16069195]\n",
      " [-0.22356299  0.37934533]\n",
      " [ 0.06353563  0.42780027]\n",
      " [ 0.42908934  0.43098518]\n",
      " [-0.2863296  -0.23091573]\n",
      " [-0.02367118 -0.0383307 ]\n",
      " [-0.09761468 -0.14817584]\n",
      " [-0.30554348 -0.21575972]\n",
      " [-0.33133447 -0.13393629]\n",
      " [ 0.11791775  0.42493823]]\n",
      "Variable:  fcon1/bias:0\n",
      "Shape:  (2,)\n",
      "[0. 0.]\n"
     ]
    }
   ],
   "source": [
    "#looking at the various trainable variables\n",
    "variables_names = [v.name for v in tf.trainable_variables()]\n",
    "values = sess.run(variables_names)\n",
    "for k, v in zip(variables_names, values):\n",
    "    print (\"Variable: \", k)\n",
    "    print (\"Shape: \", v.shape)\n",
    "    print (v)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training the model\n",
    "<p>Here we create a large batch for evaluation purposes then we train the model printing out the training and evaluation results periodically.  It is worth noting that for the default metaparamters defined above sometimes the model gets stuck in a local loss minimum where either the predictions are always 1 or 0 or the predictions are actually the opposite of the expected result.  Starting from scratch and/or changing the training rate can fix this.  Also, the number of steps to get 100% accuracy can vary wildly for the same learning rate.</p>\n",
    "<p> All of this can be improved by increasing the number of filters.  This speeds up both training and reduces chances of getting stuck in a local mazimum, but increases the size of the model, which should be as small as possible for the unit test.  Based upon testing the resulting size of the model it turns out numfilters = 15 minimize the size and maximizes stability.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "step=0                #initialling my step count so I can use my training block multiple times without \n",
    "                      #    resetting the count.\n",
    "trainres=[]           # Saving results along the way for plotting later\n",
    "evelres=[]            # Saving results along the way for plotting later"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "ex,ey=create_batch(eval_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training step     0: loss = 0.58613 accuracy = 0.54000\n",
      "EVAL     0: loss = 0.70886 accuracy = 0.44800\n",
      "Training step    25: loss = 0.43587 accuracy = 0.84000\n",
      "Training step    50: loss = 0.38358 accuracy = 0.88000\n",
      "Training step    75: loss = 0.23036 accuracy = 0.98000\n",
      "Training step   100: loss = 0.20598 accuracy = 1.00000\n",
      "EVAL   100: loss = 0.21641 accuracy = 0.99000\n",
      "Training step   125: loss = 0.21126 accuracy = 0.98000\n",
      "Training step   150: loss = 0.21569 accuracy = 1.00000\n",
      "Training step   175: loss = 0.23280 accuracy = 0.96000\n",
      "Training step   200: loss = 0.17887 accuracy = 1.00000\n",
      "EVAL   200: loss = 0.18435 accuracy = 0.99200\n",
      "Training step   225: loss = 0.20992 accuracy = 1.00000\n",
      "Training step   250: loss = 0.17607 accuracy = 1.00000\n",
      "Training step   275: loss = 0.17850 accuracy = 1.00000\n",
      "Training step   300: loss = 0.21766 accuracy = 0.92000\n",
      "EVAL   300: loss = 0.17304 accuracy = 0.99600\n",
      "Training step   325: loss = 0.15414 accuracy = 1.00000\n",
      "Training step   350: loss = 0.17313 accuracy = 1.00000\n",
      "Training step   375: loss = 0.16446 accuracy = 1.00000\n",
      "Training step   400: loss = 0.15024 accuracy = 1.00000\n",
      "EVAL   400: loss = 0.16558 accuracy = 0.99000\n",
      "Training step   425: loss = 0.17004 accuracy = 0.98000\n",
      "Training step   450: loss = 0.15312 accuracy = 1.00000\n",
      "Training step   475: loss = 0.15868 accuracy = 1.00000\n",
      "Training step   500: loss = 0.16762 accuracy = 1.00000\n",
      "EVAL   500: loss = 0.16135 accuracy = 0.99200\n",
      "Training step   525: loss = 0.14295 accuracy = 1.00000\n",
      "Training step   550: loss = 0.15175 accuracy = 1.00000\n",
      "Training step   575: loss = 0.17808 accuracy = 0.98000\n",
      "Training step   600: loss = 0.15602 accuracy = 1.00000\n",
      "EVAL   600: loss = 0.15754 accuracy = 0.99400\n",
      "Training step   625: loss = 0.17074 accuracy = 0.98000\n",
      "Training step   650: loss = 0.16756 accuracy = 1.00000\n",
      "Training step   675: loss = 0.15593 accuracy = 1.00000\n",
      "Training step   700: loss = 0.17631 accuracy = 1.00000\n",
      "EVAL   700: loss = 0.15284 accuracy = 0.99200\n",
      "Training step   725: loss = 0.13856 accuracy = 1.00000\n",
      "Training step   750: loss = 0.18030 accuracy = 0.96000\n",
      "Training step   775: loss = 0.14888 accuracy = 1.00000\n",
      "Training step   800: loss = 0.18766 accuracy = 0.98000\n",
      "EVAL   800: loss = 0.14997 accuracy = 0.99600\n",
      "Training step   825: loss = 0.16449 accuracy = 0.98000\n",
      "Training step   850: loss = 0.16190 accuracy = 0.98000\n",
      "Training step   875: loss = 0.15237 accuracy = 1.00000\n",
      "Training step   900: loss = 0.19313 accuracy = 1.00000\n",
      "EVAL   900: loss = 0.14780 accuracy = 0.99200\n",
      "Training step   925: loss = 0.17830 accuracy = 1.00000\n",
      "Training step   950: loss = 0.17351 accuracy = 0.98000\n",
      "Training step   975: loss = 0.15952 accuracy = 1.00000\n",
      "Training step  1000: loss = 0.17279 accuracy = 0.98000\n",
      "EVAL  1000: loss = 0.14658 accuracy = 0.99600\n",
      "Training step  1025: loss = 0.13590 accuracy = 1.00000\n",
      "Training step  1050: loss = 0.13560 accuracy = 1.00000\n",
      "Training step  1075: loss = 0.13202 accuracy = 1.00000\n",
      "Training step  1100: loss = 0.13394 accuracy = 1.00000\n",
      "EVAL  1100: loss = 0.14538 accuracy = 0.99400\n",
      "Training step  1125: loss = 0.16967 accuracy = 1.00000\n",
      "Training step  1150: loss = 0.17313 accuracy = 1.00000\n",
      "Training step  1175: loss = 0.14891 accuracy = 1.00000\n",
      "Training step  1200: loss = 0.14862 accuracy = 1.00000\n",
      "EVAL  1200: loss = 0.14480 accuracy = 0.99600\n",
      "Training step  1225: loss = 0.14566 accuracy = 1.00000\n",
      "Training step  1250: loss = 0.13976 accuracy = 1.00000\n",
      "Training step  1275: loss = 0.14717 accuracy = 1.00000\n",
      "Training step  1300: loss = 0.15759 accuracy = 1.00000\n",
      "EVAL  1300: loss = 0.14365 accuracy = 0.99200\n",
      "Training step  1325: loss = 0.15188 accuracy = 1.00000\n",
      "Training step  1350: loss = 0.15147 accuracy = 1.00000\n",
      "Training step  1375: loss = 0.14468 accuracy = 1.00000\n",
      "Training step  1400: loss = 0.14385 accuracy = 1.00000\n",
      "EVAL  1400: loss = 0.14378 accuracy = 0.99400\n",
      "Training step  1425: loss = 0.14460 accuracy = 1.00000\n",
      "Training step  1450: loss = 0.14565 accuracy = 1.00000\n",
      "Training step  1475: loss = 0.12744 accuracy = 1.00000\n",
      "Training step  1500: loss = 0.15052 accuracy = 0.98000\n",
      "EVAL  1500: loss = 0.14245 accuracy = 0.99200\n",
      "Training step  1525: loss = 0.14147 accuracy = 1.00000\n",
      "Training step  1550: loss = 0.13878 accuracy = 1.00000\n",
      "Training step  1575: loss = 0.13959 accuracy = 1.00000\n",
      "Training step  1600: loss = 0.15934 accuracy = 1.00000\n",
      "EVAL  1600: loss = 0.14194 accuracy = 0.99200\n",
      "Training step  1625: loss = 0.15742 accuracy = 0.98000\n",
      "Training step  1650: loss = 0.15987 accuracy = 0.98000\n",
      "Training step  1675: loss = 0.15515 accuracy = 1.00000\n",
      "Training step  1700: loss = 0.14067 accuracy = 1.00000\n",
      "EVAL  1700: loss = 0.14189 accuracy = 0.99600\n",
      "Training step  1725: loss = 0.13693 accuracy = 1.00000\n",
      "Training step  1750: loss = 0.13698 accuracy = 1.00000\n",
      "Training step  1775: loss = 0.13315 accuracy = 1.00000\n",
      "Training step  1800: loss = 0.13444 accuracy = 1.00000\n",
      "EVAL  1800: loss = 0.14099 accuracy = 0.99400\n",
      "Training step  1825: loss = 0.14646 accuracy = 1.00000\n",
      "Training step  1850: loss = 0.14200 accuracy = 1.00000\n",
      "Training step  1875: loss = 0.14024 accuracy = 1.00000\n",
      "Training step  1900: loss = 0.13113 accuracy = 1.00000\n",
      "EVAL  1900: loss = 0.14064 accuracy = 0.99200\n",
      "Training step  1925: loss = 0.13708 accuracy = 1.00000\n",
      "Training step  1950: loss = 0.15684 accuracy = 1.00000\n",
      "Training step  1975: loss = 0.14709 accuracy = 1.00000\n",
      "Training step  2000: loss = 0.13718 accuracy = 1.00000\n",
      "EVAL  2000: loss = 0.14159 accuracy = 0.99400\n"
     ]
    }
   ],
   "source": [
    "if step:\n",
    "    train_steps+=step\n",
    "\n",
    "for step in range(step,train_steps):\n",
    "    \n",
    "    xb,yb=create_batch(batchsize)\n",
    "\n",
    "    feed_dict={X:xb,Y:yb}\n",
    "    _,(ac,ac_op),lo,pr=sess.run([train_step,accuracy,loss,pred],feed_dict=feed_dict)\n",
    "    \n",
    "    if step%pstep==0:\n",
    "        ac=1-sum(np.fabs(pr-yb))/len(yb)\n",
    "        s=\"Training step {:5d}: loss = {:7.5f} accuracy = {:7.5f}\".format(step,lo,ac)\n",
    "        print(s)\n",
    "        trainres.append([step,lo,ac])\n",
    "\n",
    "    if step%estep==0:\n",
    "        feed_dict={X:ex,Y:ey}#,lr:1e-3}\n",
    "        (ac,ac_op),lo,pr2=sess.run([accuracy,loss,pred],feed_dict=feed_dict)\n",
    "        ac2=1-sum(np.fabs(pr2-ey))/len(ey)\n",
    "        print(\"EVAL {:5d}: loss = {:7.5f} accuracy = {:7.5f}\".format(step,lo,ac2))\n",
    "        evelres.append([step,lo,ac2])\n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted Output of Evaluation set:\n",
      "[1 1 0 1 1 1 1 1 0 1 1 0 1 1 0 1 1 0 1 1 0 0 0 1 1 1 0 1 0 0 1 1 1 1 0 1 0\n",
      " 0 1 1 1 0 1 1 0 0 0 1 0 1 1 0 1 0 0 0 0 1 1 0 0 1 0 1 0 1 1 0 1 0 1 1 0 0\n",
      " 1 0 1 0 1 1 1 0 1 1 0 0 1 0 1 0 0 1 0 0 1 1 1 1 0 0 1 0 1 0 0 1 0 1 1 1 1\n",
      " 0 1 0 1 0 0 1 1 1 1 0 1 0 1 0 1 1 0 0 1 1 1 1 1 0 0 0 0 0 1 1 1 1 0 1 0 0\n",
      " 1 1 1 1 0 0 1 0 1 1 1 0 1 1 0 1 1 1 0 1 1 1 1 0 0 1 0 0 1 1 1 0 1 1 0 0 1\n",
      " 1 0 0 1 0 1 1 1 1 0 0 1 0 0 0 1 1 1 1 0 0 1 0 0 1 0 0 1 1 1 1 1 0 0 0 0 0\n",
      " 1 1 0 1 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 1 1 1 0 0 1 0 0 1 1 1 0 1 0 0 0\n",
      " 0 1 0 0 0 1 1 1 1 1 1 0 0 1 1 0 0 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1\n",
      " 1 1 0 1 1 0 0 0 0 1 1 0 1 1 0 1 0 1 1 1 1 0 1 1 0 1 1 0 1 1 1 0 1 1 0 1 1\n",
      " 1 1 1 1 1 0 1 1 0 1 1 0 0 1 0 0 0 1 0 1 0 1 0 1 1 0 0 1 1 1 1 1 0 1 1 0 0\n",
      " 0 1 1 1 1 1 0 0 0 1 1 1 0 1 1 1 0 0 0 1 1 1 0 0 0 0 1 1 0 1 1 0 0 1 0 1 0\n",
      " 0 0 1 1 0 0 0 0 1 1 0 0 0 1 1 1 0 1 1 0 1 1 1 1 1 1 1 0 1 1 0 0 1 0 1 1 0\n",
      " 0 0 1 1 0 1 1 0 0 1 0 1 1 0 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 1 1 1 1 1 0 1 0\n",
      " 0 0 1 0 0 1 0 1 0 1 0 0 1 0 0 0 0 1 1]\n",
      "Evalutation Set Labels:\n",
      "[1 1 0 1 1 1 1 1 0 1 1 0 1 1 0 1 1 0 1 1 0 0 0 1 1 1 0 1 0 0 1 1 1 1 0 1 0\n",
      " 0 1 1 1 0 1 1 0 0 0 1 0 1 1 0 1 0 0 0 0 1 1 0 0 1 0 1 0 1 1 0 1 0 1 1 0 0\n",
      " 1 0 1 0 1 1 1 0 1 1 0 0 1 0 1 0 0 1 0 0 1 1 1 1 0 0 1 0 1 0 0 1 0 1 1 1 1\n",
      " 0 1 0 1 0 0 1 1 1 1 0 1 0 1 0 1 1 0 0 1 1 1 1 1 0 0 0 0 0 1 1 1 1 0 1 0 0\n",
      " 1 1 1 1 0 0 1 0 1 1 1 0 1 1 0 1 1 1 0 1 1 1 1 0 0 1 0 0 1 1 1 0 1 1 0 0 1\n",
      " 1 0 0 1 0 1 1 1 1 0 0 1 0 0 0 1 1 1 1 0 0 1 0 0 1 0 0 1 1 1 1 1 0 0 0 0 0\n",
      " 1 1 0 1 1 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 1 1 1 0 0 1 0 0 1 1 1 0 1 0 0 0\n",
      " 0 1 0 0 0 1 1 1 1 1 1 0 0 1 1 0 0 0 1 0 1 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 1\n",
      " 1 1 0 1 1 0 0 0 0 1 1 0 1 1 0 1 0 1 1 1 1 0 1 1 0 1 1 0 1 1 1 0 1 1 0 1 1\n",
      " 1 1 1 1 1 0 1 1 0 1 1 0 0 1 0 0 0 1 0 1 0 1 0 1 1 0 0 1 1 1 1 1 0 1 1 0 0\n",
      " 0 1 1 1 1 1 0 0 0 1 1 1 0 1 1 1 0 0 0 1 1 1 0 0 0 0 0 1 0 1 1 0 0 1 0 1 0\n",
      " 0 0 1 1 0 0 0 0 1 1 0 0 0 1 1 1 0 1 1 0 1 1 1 1 1 1 1 0 1 1 0 0 1 0 1 1 0\n",
      " 0 0 1 1 0 1 1 0 0 1 0 1 1 0 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 1 1 1 1 1 0 1 0\n",
      " 0 0 1 0 0 1 0 1 0 1 0 0 1 0 0 0 0 0 1]\n"
     ]
    }
   ],
   "source": [
    "print(\"Predicted Output of Evaluation set:\")\n",
    "print(pr2)\n",
    "print(\"Evalutation Set Labels:\")\n",
    "print(ey)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Printing Loss and Accuracy for training and Evaluation sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "tr=np.array(trainres)\n",
    "ev=np.array(evelres)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><b>Note</b>: that training is done with new data (i.e. not previously seen by the model) at every step without epocs.  This means the training and evaluation data are comparable.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzs3XecVOX1+PHPmdnZXlk6C1KkSBcXFSUKYhS7iT3R2GI3aopRY6LGX0yMMZoY+dqiaDT2qIBiV1As9N47LCxsgS1sL+f3x727zPbLsrO7sOf9es1r5t65986ZWZhnnnYeUVWMMcYYAF9bB2CMMab9sELBGGNMNSsUjDHGVLNCwRhjTDUrFIwxxlSzQsEYY0w1KxSMaQdE5Kci8klLH2vMgRKbp2AaIiJbgJ+r6mdtHUt7JiIvAmmq+vu2jiVUROQB4EhVvbytYzGhZTUFc9gSkbC2jgHaTxzGeGGFgmkWEblORDaIyB4RmS4iPd39IiKPi0iGiOSKyDIRGe4+d6aIrBKRfBHZISK/qee6ESKSU3WOu6+LiBSJSFd3+2wRWeIe962IjAw6douI3CUiy4ACEQlzt3e4r7tWRCa5x74oIn8KOneCiKQFbdd7Xq14rwd+CvxWRPaJyIxG4rhbRDa611slIj8Kus5VIjInaFtF5EYRWS8ie0VkiohIM471i8jfRSRLRDaLyK3u8fUWVPW9ZxGZDPwOuMR9j0vdYxNE5HkRSXfP+ZOI+INi/EZE/uX+O1hT3+dn2iFVtZvd6r0BW4BT69l/CpAFjAEigH8BX7nPnQ4sBBIBAY4CerjPpQM/cB8nAWMaeN0XgIeCtm8BPnIfjwEygOMAP3ClG2dEUMxLgN5AFDAY2A70dJ/vCwxwH78I/CnodSbgNAPR2Hn1xFvjOvXF4e67COiJ82PsEqAg6LO5CpgTdL4C77ufYx8gE5jcjGNvBFYBKe5n/pl7fFg976Oxz+oB4JVax78HPAPEAF2BecANQTGWA78EAu77zQU6tfW/a7s1frOagmmOnwIvqOoiVS0B7gHGiUhfoAyIA4bg9FmtVtV097wyYKiIxKvqXlVd1MD1XwUuC9r+ibsP4DrgGVWdq6oVqvoSUAIcH3T8E6q6XVWLgAqcgmuoiARUdYuqbvTwHpt7XrDgOFDVt1R1p6pWquobwHrg2EbOf1hVc1R1G/AlMLoZx14M/FNV01R1L/BwI9fw/J5FpBtwBnCHqhaoagbwOHBp0GEZwD9Utcx9v2uBsxp5fdMOWKFgmqMnsLVqQ1X3AdlAL1X9AngSmALsFpFnRSTePfQC4Exgq4jMFpFxDVz/CyBKRI4TkSNwvuDedZ87Avi123SUIyI5OL/Gewadvz0otg3AHTi/dDNE5PWqpq7GNPe8WrYHb4jIz4KavXKA4UDnRs7fFfS4EIhtxrE9a8VRI6ZgB/iej8CpAaQHvZ9ncGoMVXaoavBIlq3U/DuZdsgKBdMcO3G+FAAQkRggGdgBoKpPqOoxwDBgEHCnu3++qp6H88XxHvBmfRdX1Ur3uctwagnvq2q++/R2nKalxKBbtKq+FnyJWtd7VVXHuzEr8Ff3qQIgOujQ7h7PqxNyU/vdwu054FYgWVUTgRU4TWyhlI7TdFSld2MHN/Kea7/H7Tg1tM5Bf4d4VR0WdEyvqr4NVx+cfzumHbNCwTQlICKRQbcwnKacq0VktIhEAH8G5qrqFhEZ6/7CD+B86RYDFSISLs74+gRVLQPycJorGvIqTjv0T9nfdATOF+uN7muIiMSIyFkiElffRURksIic4sZZDFQ1KYHT5n+miHQSke44v5K9nFfbbqB/I+8FnHZ3xWnvR0SuxqkphNqbwO0i0ktEEoG7Gjqwife8G+grIj4At0nwE+DvIhIvIj4RGSAiJwddsitwm4gEROQinP6lmS3+Dk2LskLBNGUmzpdD1e0BVf0c+APwP5xfogPY35Ycj/PFvRenuSAbeNR97gpgi4jk4XSANjjmXVXn4hQqPYEPg/YvwOlXeNJ9jQ04nZoNicBpR8/CaWLpijOSBuBlYClOp/AnwBsez6vteZx2+BwRea+B97MK+DvwHc4X7Ajgm0bibinP4by3ZcBinL9nOfUXcI2957fc+2wRqeoL+hkQjtORvRd4G+gRdL25wED3eg8BF6pqNoCIPC0iT7fA+zMtzCavGdOBiMgZwNOqekSTBx/c61yFM/FxfChfx7Q8qykYcxgTkShx5oeEiUgv4H72d9obU4cVCsYc3gT4I07zzmJgNXBfm0Zk2jVrPjLGGFPNagrGGGOqHXKJujp37qx9+/Zt6zCMMeaQsnDhwixV7dLUcYdcodC3b18WLFjQ1mEYY8whRUS2Nn2UNR8ZY4wJYoWCMcaYalYoGGOMqXbI9SkYY4xXZWVlpKWlUVxc3NahtJrIyEhSUlIIBALNOt8KBWPMYSstLY24uDj69u1LzYSthydVJTs7m7S0NPr169esa1jzkTHmsFVcXExycnKHKBAARITk5OSDqhlZoWCMOax1lAKhysG+345TKGz9Dj57ACythzHGNCikhYKITBaRtSKyQUTuruf5x93lCZeIyDp3Sb/Q2LkI5jwORXtD9hLGGBMsOzub0aNHM3r0aLp3706vXr2qt0tLSz1d4+qrr2bt2rUhjnS/kHU0i4gfZ53eHwJpwHwRme4uNgKAqv4y6PhfAEeHKh5i3KVjCzIhulPIXsYYY6okJyezZMkSAB544AFiY2P5zW9+U+MYVUVV8fnq/40+derUkMcZLJQ1hWOBDaq6SVVLgdeB8xo5/jLgtUaePzixbsqPfRkhewljjPFiw4YNDB8+nBtvvJExY8aQnp7O9ddfT2pqKsOGDePBBx+sPnb8+PEsWbKE8vJyEhMTufvuuxk1ahTjxo0jI6Plv89COSS1F87i3lXSgOPqO9Bd1Lwf8EUDz18PXA/Qp0+f5kUT282537e7eecbYw5pf5yxklU781r0mkN7xnP/OcOade6qVauYOnUqTz/trEr68MMP06lTJ8rLy5k4cSIXXnghQ4cOrXFObm4uJ598Mg8//DC/+tWveOGFF7j77jot8wcllDWF+rrAG+rlvRR4W1XrXRhdVZ9V1VRVTe3Spckkf/ULbj4yxpg2NmDAAMaOHVu9/dprrzFmzBjGjBnD6tWrWbVqVZ1zoqKiOOOMMwA45phj2LJlS4vHFcqaQhrQO2g7BdjZwLGXAreEMBaISgLxW/ORMR1Uc3/Rh0pMTEz14/Xr1/PPf/6TefPmkZiYyOWXX17vXIPw8PDqx36/n/Ly8haPK5Q1hfnAQBHpJyLhOF/802sfJCKDgSTguxDGAj4fxHaFAisUjDHtS15eHnFxccTHx5Oens7HH3/cZrGErKagquUicivwMeAHXlDVlSLyILBAVasKiMuA17U11gWN6WI1BWNMuzNmzBiGDh3K8OHD6d+/PyeeeGKbxXLIrdGcmpqqzV5k55ULoCALbpjdskEZY9ql1atXc9RRR7V1GK2uvvctIgtVNbWpczvOjGZwOputo9kYYxrUsQqF2K5O89EhVjsyxpjW0mifgohEAmcDPwB6AkXACuADVV0Z+vBaWGxXqCxzUl3YrGZjjKmjwUJBRB4AzgFmAXOBDCASGAQ87BYYv1bVZaEPs4VYqgtjjGlUYzWF+ar6QAPPPSYiXYFmTi9uI8GpLroMbttYjDGmHWqwT0FVPwAQkeENPJ+hqs0cBtRGLNWFMcY0yktH89MiMk9EbhaRxJBHFEqW6sIY08r8fn91uuzRo0fz8MMPN+s6EyZMoNnD8Q9Ak5PXVHW8iAwErgEWiMg8YKqqfhry6FqapbowxrSyqKio6vTZhwJPQ1JVdT3we+Au4GTgCRFZIyI/DmVwLc5SXRhj2oEPP/yQiy++uHp71qxZnHPOOQDcdNNN1Sm077///laPrcmagoiMBK4GzgI+Bc5R1UUi0hMnX9E7oQ2xhVmqC2M6pg/vhl3LW/aa3UfAGY03BxUVFTF69Ojq7XvuuYcLLriAG264gYKCAmJiYnjjjTe45JJLAHjooYfo1KkTFRUVTJo0iWXLljFy5MiWjbsRXnIfPQk8B/xOVYuqdqrqThH5fcgiC5WqCWzGGNMKGmo+mjx5MjNmzODCCy/kgw8+4JFHHgHgzTff5Nlnn6W8vJz09HRWrVrVvgoFVT3JzXI6REQUWOuupIaqvhzqAFtcTFfIWN3WURhjWlsTv+hb2yWXXMKUKVPo1KkTY8eOJS4ujs2bN/Poo48yf/58kpKSuOqqq+pNoR1KTfYpiMiZwEbgCZxawwYROSPUgYWMpbowxrQDEyZMYNGiRTz33HPVTUd5eXnExMSQkJDA7t27+fDDD1s9Li/NR48BE1V1A4CIDAA+AFo/2pZgqS6MMa2odp/C5MmTefjhh/H7/Zx99tm8+OKLvPTSSwCMGjWKo48+mmHDhrVZCm0vhUJGVYHg2oST8uLQZKkujDGtqKKi3lWGAXjyySd58skna+x78cUX6z121qxZLRhVw7wUCitFZCbwJs4ayxcB86uGo6rqoTX6yFJdGGNMg7wUCpHAbpz5CQCZQCecZHnKoTYktSrVhc1VMMaYOryMPrq6NQJpNVXNRzYs1ZgOQVURkbYOo9Uc7GqaXkYfpYjIuyKSISK7ReR/IpJyUK/alizVhTEdRmRkJNnZ2Qf9RXmoUFWys7OJjIxs9jW8NB9NBV7F6UsAuNzd98Nmv2pbslQXxnQYKSkppKWlkZnZcZJgRkZGkpLS/N/tXgqFLqo6NWj7RRG5o9mv2B5YqgtjOoRAIEC/fv3aOoxDipeEeFkicrmI+N3b5UB2qAMLKUt1YYwx9fJSKFwDXAzsAtKBC919h66YrramgjHG1KPR5iMR8QMXqOq5rRRP6whOddGBRiUYY0xTGq0pqGoFcF4rxdJ6glNdGGOMqeal+egbEXlSRH4gImOqbl4uLiKTRWStiGwQkbsbOOZiEVklIitF5NUDir65bFlOY4ypl5fRRye49w8G7VPglMZOcpuepuAMXU3DSY0xXVVXBR0zELgHOFFV94pI1wMJvtks1YUxxtTLS6FwrapuCt4hIv09nHcssKHqXBF5HacpalXQMdcBU1R1L4Cqts6QIEt1YYwx9fLSfPR2Pfve8nBeL2B70Haauy/YIGCQiHwjIt+LyOT6LiQi14vIAhFZ0CKTUCzVhTHG1KvBmoKIDAGGAQlVGVFd8ThJ8ppS37Ce2nPNw4CBwAQgBfhaRIarak6Nk1SfBZ4FSE1NPfj56pbqwhhj6tVY89Fg4GwgEScjapV8nGafpqQBvYO2U4Cd9RzzvaqWAZtFZC1OITHfw/Wbz1JdGGNMvRosFFR1GjBNRMap6nfNuPZ8YKCI9AN2AJcCP6l1zHvAZTipMzrjNCdtojXEdIF9NvrIGGOCeelo3iAivwP6Bh+vqo3OalbVchG5FfgY8AMvqOpKEXkQWKCq093nThORVUAFcKeqhiSFxjuL0pj6zRbevfkEwvxuTWHf7lC8lDHGHLK8FArTgK+Bz3C+uD1T1ZnAzFr77gt6rMCv3FtI5RWVsXxHLrlFZSTHRjidzRmrQ/2yxhhzSPFSKESr6l0hjyTEkmLCAdhb6BYKlurCGGPq8DIk9X0ROTPkkYRYUnRVoVDq7LBUF8YYU4eXQuF2nIKhWETyRCRfRPJCHVhLqy4UCtxCwVJdGGNMHV7WaI5rjUBCLTE6AEBOYZmzw1JdGGNMHV7WaBZ3kZ0/uNu9ReTY0IfWsvb3KVQ1H1mqC2OMqc1L89H/AePYP8dgH06iu0NKTLifgF/YW1VTsFQXxhhTh5fRR8ep6hgRWQzgZjMND3FcLU5ESIwOJ6eqpmCpLowxpg4vNYUyNw22AohIF6AypFGFSFJ0YH/zkaW6MMaYOrwUCk8A7wJdReQhYA7w55BGFSKJ0eH7m4/AUl0YY0wtXkYf/VdEFgKTcDKfnq+qh+RU4KToAJuzCvbvsFQXxhhTg5c+BVR1DbAmxLGEXFJ0OIsKg7JyW6oLY4ypwUvz0WGjqqPZSblEzVQXxhhjOlahkBQdoKxCKSh18/pZqgtjjKnBy+S1GBHxuY8Hici5IhIIfWgtz1JdGGNM47zUFL4CIkWkF/A5cDXwYiiDCpW6s5ptApsxxgTzUiiIqhYCPwb+pao/AoaGNqzQSHLzH1UPS60qFGyugjHGAB4LBREZB/wU+MDd52nUUnuT6DYfVc9qtlQXxhhTg5dC4Q7gHuBddznN/sCXoQ0rNKprCgWW6sIYY+rjZfLabGA2gNvhnKWqt4U6sFBIiKrVfGSpLowxpgYvo49eFZF4EYkBVgFrReTO0IfW8sL8PuIjw/Y3H4GlujDGmCBemo+GqmoecD4wE+gDXBHSqEIoKaZW/iNLdWGMMdW8FAoBd17C+cA0VS3DzZh6KHKS4gXXFLraPAVjjHF5KRSeAbYAMcBXInIEcMit0VwlKTqwf0lOcPsUMi3VhTHG4KFQUNUnVLWXqp6pjq3AxFaILSSSatcUYrtCRSkU5zR8kjHGdBBeOpoTROQxEVng3v6OU2s4JCXWrinYXAVjjKnmpfnoBSAfuNi95QFTvVxcRCaLyFoR2SAid9fz/FUikikiS9zbzw8k+OZIig5nX0k5peXu4nGW6sIYY6p5mZk8QFUvCNr+o4gsaeokdwnPKcAPgTRgvohMV9VVtQ59Q1Vv9RzxQarKf5RTWErX+EhLdWGMMUG81BSKRGR81YaInAgUeTjvWGCDqm5S1VLgdeC85oXZcurkP7LmI2OMqealpnAj8B8RSXC39wJXejivF7A9aDsNOK6e4y4QkZOAdcAvVXV77QNE5HrgeoA+ffp4eOmGVafPLrRUF8YYU1ujNQU3rcVgVR0FjARGqurRqrrMw7Wlnn21x33OAPqq6kjgM+Cl+i6kqs+qaqqqpnbp0sXDSzcs0a0pVM9qtlQXxhhTrdFCQVUrgVvdx3nuzGav0oDeQdspwM5a189W1RJ38zngmAO4frPsrykEj0CyVBfGGAPe+hQ+FZHfiEhvEelUdfNw3nxgoIj0E5Fw4FJgevABItIjaPNcYLXnyJupTvMRWKoLY4xxeelTuMa9vyVonwL9GztJVctF5FbgY8APvOCm3n4QWKCq04HbRORcoBzYA1x1gPEfsKhwPxFhvrpzFTJCXh4ZY0y75yV1dr/mXlxVZ+Ik0Qved1/Q43tw1mpoVUnR4fvXVICaqS6kvq4QY4zpGLzMaL5FRBKDtpNE5ObQhhVaidGBuplSLdWFMcZ46lO4TlWrvy1VdS9wXehCCr2k6PBaayrYXAVjjAFvhYJPZH+bijtTOTx0IYVeUkygbkczWKFgjOnwvHQ0fwy8KSJP43Qw3wh8FNKoQiwxOrxu+mywuQrGmA7PS6FwF85s4ptwJqR9Avw7lEGFWlK0U1OorFR8PglqPrK5CsaYjs3L6KNK4Gn3dlhIig6nUiG/uJyE6EBQqgubq2CM6di89CkcdupMYLNUF8YYA3TUQiGmKlNq8AgkS3VhjDFe5ilE1rOvc2jCaR2J0VVrKtTqbLbmI2NMB+elpjBfRI6v2hCRC4BvQxdS6NWb/yjGndVsjDEdmJfRRz8BXhCRWUBPIBk4JZRBhVqdhXbAUl0YYwzeRh8tF5GHgJdx1mo+SVXTQh5ZCMVHBvAJNWc1B6e6iEpqu+CMMaYNeelTeB64A2eRnauBGSJyS+NntW8+n5AQFajbfAQ2q9kY06F56VNYAUxU1c2q+jFwPDAmtGGFXlJ0eN3mI7BCwRjToXlpPnq81nYucG3IImolidGBus1HYHMVjDEdWpOFgogMBP4CDAWqh6eqaqOL7LR3SdHh7Mwt3r/DUl0YY4yn5qOpwFM4q6NNBP6D0+l8SEtJimJbdgEVlerssFQXxhjjqVCIUtXPAVHVrar6AIf4kFSAUb0TKSitYH1GvrPDUl0YY4ynQqFYRHzAehG5VUR+BHQNcVwhN6q3s5jckm1Bq61ZqgtjTAfnpVC4A4gGbgOOAa4ArgxlUK2hX3IM8ZFhLE0LKhSspmCM6eC8jD6a7z7chzNP4bDg8wmjeieyuEZNoStkrG67oIwxpo15mbyWKiLvisgiEVlWdWuN4ELt6N6JrNudT0FJubMjONWFMcZ0QF5yH/0XuBNYDlSGNpzWNbpPIpUKy3fkcnz/ZEt1YYzp8LwUCpmqOj3kkbSBUSlOZ/PS7TlOoRCc6sIKBWNMB+SlULhfRP4NfA6UVO1U1XdCFlUrSY6NoHenKJZsd/sVglNddBncdoEZY0wb8TL66GpgNDAZOMe9ne3l4iIyWUTWisgGEbm7keMuFBEVkVQv121Jo3sn1S0UbASSMaaD8lJTGKWqIw70wiLiB6YAPwTScBbrma6qq2odF4cz3HXugb5GSxjdO5EZS3eyO6+YbpbqwhjTwXmpKXwvIkObce1jgQ2quklVS4HXgfPqOe7/AY8AxfU8F3KjeycAOENTLdWFMaaD81IojAeWuM1Ay0Rkucchqb2A7UHbae6+aiJyNNBbVd9v7EIicr2ILBCRBZmZLfsrfljPBMJ84kxis1QXxpgOzkvz0eRmXru+NS2rJwC4qTMeB65q6kKq+izwLEBqamqLTiKIDPg5qkf8/nQXlurCGNOBeakp/MlNhFd9A/7k4bw0oHfQdgqwM2g7DhgOzBKRLTiL90xvm87mRJal5TgZU62mYIzpwLwUCsOCN9wO5GM8nDcfGCgi/UQkHLgUqJ7voKq5qtpZVfuqal/ge+BcVV3gOfoWMtrNmLohY58zV8FWXzPGdFANFgoico+I5AMjRSTPveUDGcC0pi6squXArcDHwGrgTVVdKSIPisi5LRR/i6jKmLp0e46lujDGdGiN9Sl8pap/EZGHVbXBOQaNUdWZwMxa++5r4NgJzXmNltC/cwxxkWEsScvh4u6W6sIY03E11nz0hHt/WmsE0pZ8PqFPp2h25xbXTHVhjDEdTGM1hTIRmQr0EpEnaj+pqreFLqzWlxgdYG9hqaW6MMZ0aI0VCmcDp+IsvbmwdcJpO4nR4aTn5EFsN2eHjUAyxnRADRYKqpoFvC4iq1V1aSvG1CaSogPkFJUFNR/ZXAVjTMfjZUhqtrvIToaI7BaR/4lISsgja2WJUeHkFJZSGZloqS6MMR2Wl0JhKs78gp44aSpmuPsOK4nRASoV8ksqbQKbMabD8lIodFXVqapa7t5eBLqEOK5WlxQdDuB0NluqC2NMB+WlUMgUkctFxO/eLgeyQx1Ya0uKCQA4/QpWUzDGdFBeCoVrgIuBXe7tQnffYSUhKqimENvN5ikYYzqkJrOkquo2oF2lpQiFpGi3plDVfFSV6kLqS/ZqjDGHpyZrCiLyiIjEi0hARD4XkSy3CemwUtWnkFPoNh9VpbowxpgOxEvz0WmqmoczmS0NGATcGdKo2kB8VAAR2FtocxWMMR2Xl0Ih4N6fCbymqntCGE+b8fuE+MiA03xUnerC5ioYYzoWLyuvzRCRNUARcLOIdKGN1lMOtaTogFNTqCoUbASSMaaDabKm4KbNHgekqmoZUAicF+rA2kJCdLjb0WzNR8aYjslLTQFV3Rv0uAAoCFlEbSgpOkD2vlJnHQVLdWGM6YC89Cl0GEnR4c48BZ/PJrAZYzokKxSCJEYHyC0sczYs1YUxpgPyMk9B3DQX97nbfUTk2NCH1voSo8LJLymnrMKS4hljOiYvNYX/w+lovszdzgemhCyiNlSd/6iwzFJdGGM6JC+FwnGqegvuMFS30zk8pFG1kcTqWc21Ul0YY0wH4aVQKBMRP6AA7jyFypBG1UYSo2plSrVUF8aYDsZLofAE8C7QVUQeAuYAfw5pVG2kek2FApurYIzpmLxkSf2viCwEJgECnK+qq0MeWRtIjA7qU0gOSnXRZVAbRmWMMa3Hy+ijAcBmVZ0CrAB+KCKJIY+sDVQXCkWllurCGNMheWk++h9QISJHAv8G+gGverm4iEwWkbUiskFE7q7n+RtFZLmILBGROSIy9ICib2GxEWGE+cQypRpjOiwvhUKlqpYDPwb+qaq/BHo0dZLbOT0FOAMYClxWz5f+q6o6QlVHA48Ajx1Q9C1MREisyn9UlerCagrGmA7E6+ijy4CfAe+7+wKNHF/lWGCDqm5S1VLgdWol0nPXaagSgzvCqS0lRgecPoWqVBeW/8gY04F4KRSuxpm89pCqbhaRfsArHs7rBWwP2k5z99UgIreIyEacmsJt9V1IRK4XkQUisiAzM7TNOU767FJnw1JdGGM6mEYLBbcJ6HeqepuqvgagqptV9WEP165vceM6NQFVnaKqA4C7gN/XdyFVfVZVU1U1tUuXLh5euvmc5iM3/5GlujDGdDCNFgqqWgF0EZHmzGBOA3oHbacAOxs5/nXg/Ga8TouqUVOwVBfGmA7Gy3oKW4BvRGQ6QesoqGpTncLzgYFuc9MO4FLgJ8EHiMhAVV3vbp4FrKeN1agpBKe6kPoqPsYYc3jxUijsdG8+IM7rhVW1XERuBT4G/MALqrpSRB4EFqjqdOBWETkVKAP2Alce6BtoaYnRAUrKKykqrSAqONVFVFJbh2aMMSHnZUbzH5t7cVWdCcyste++oMe3N/faoVKd6qKwlKjguQpWKBhjOoAmCwU3Ad5vgWFAZNV+VT0lhHG1meqkeIVl9Iy1VBfGmI7Fy5DU/wJrcGYy/xGnj2F+CGNqUzXSZ1uqC2NMB+OlUEhW1eeBMlWdrarXAMeHOK42U7XQjqW6MMZ0RF46mt2hOKSLyFk4nc4poQupbSVG7e9TIKqbpbowxnQoXgqFP4lIAvBr4F9APPDLkEbVhqoypeYWWaoLY0zH42X0UVW+o1xgYmjDaXuRAT9RAb+z0A5YqgtjTIfiZT2F/iIyQ0SyRCRDRKaJSP/WCK6tOLOaLdWFMabj8dLR/CrwJtAd6Am8BbwWyqDaWkJ0OLlFwakurKZgjOkYvBQKoqovq2q5e3uFdpDiOpRq1BRiujg1BT2s37IxxgDeCoUvReRuEekrIkeIyG+BD0Skk4h0CnWAbSEpOjwoKV5QqgtjjDnMeRl9dIl7f0Ot/dfg1BiuqUCqAAAgAElEQVQOu/6FhKqFdqDmXAVLdWGMOcx5GX3UrzUCaU+SogPkFJZSWan4LNWFMaYD8TL66CIRiXMf/15E3hGRo0MfWttJig6nUiG/pNxSXRhjOhQvfQp/UNV8ERkPnA68BDwd2rDaVkJ1UrxSS3VhjOlQvBQKFe79WcBTqjoNaM5KbIeMpOqkeGVOP4KlujDGdBBeCoUdIvIMcDEwU0QiPJ53yNqfFK/UUl0YYzoUL1/uF+OsnjZZVXOATsCdIY2qjSVEBdUU4IBTXfx37la+XGM1C2PMoafB0UciEquq+1S1EHinar+qpgPpwceEPszWleQmxcvaV+LsOIBUF99tzObed1dwdJ9EJg7pGqoQjTEmJBqrKUwTkb+LyEkiElO1082FdK2IfAxMDn2Ira9TTDj9OscwbclOVNVzqoui0grufmcZAKt25lFWURnqUI0xpkU1WCio6iTgc5xJaytFJFdEsoFXcPIgXamqb7dOmK1LRLjuB/1ZviOX7zZle0518fdP1rI1u5DLju1DSXkla3flt1LExhjTMhrtU1DVmar6U1Xtq6oJqpqsqieo6kOququ1gmwLPx7Ti86x4Twze5OnVBeLtu3lhW8285Pj+nDjyc4k72Vpua0VrjHGtIjDehTRwYgM+LnqhL7MXpfJzvJYZ2cDTUgl5RX89u1ldIuP5J4zhtCnUzSJ0QGWpR18vqS5m7JZscMKF2NM67BCoRGXH38E0eF+3tng7pj7NJSXVD9fWal8uyGLG15eyIaMffz5xyOIiwwgIozolcDSg6wpVFYqt762mAffX3VQ1zHGGK+8JMTrsBKjw7lkbG/++V05V6deRcyC5yFtHrlnPsOrmyJ5ff42tmYXEh8Zxp2nD2bi4P2jjUalJPLU7I0UlVYQFe5v1uuvSs8jM7+E0vJKVBURaam3Zowx9fKS+2iAO2ENEZkgIreJSGLoQ2sfrh3fj0p8/CPiBrLPeYnCzG2EPz+B7Z9OoVtcBI9fMop5957KLROPrHHeyJQEKiqVVek1awvpuUVc8NS3bM4qaPK1Z611hsHmFpWxK6+45d6UMcY0wEvz0f+AChE5Enge6IezGluTRGSyiKwVkQ0icnc9z/9KRFaJyDIR+VxEjjig6FtBSlI0Z4/swUvfbeX4d8I5pegvbIsdyZ8Dz/Nm4hR+NDiayEDdmsCo3k65uWR7zULhzflpLNy6lzfmb2/ytWetzSQy4PyJ1thIJmNMK/BSKFSqajnwI+AfqvpLoEdTJ4mIH5gCnAEMBS4TkaG1DlsMpKrqSOBt4JEDCb613DLxSLrGRfCTY/vw9p0/YvCvP4XTHoJ1H8NTJ8CmWXXO6RYfSbf4iBqdzarKe0t2ADBzebozB6IBuYVlLNq2l4uO6Q1gw1uNMa3CS6FQJiKXAVcC77v7Ah7OOxbYoKqbVLUUeB04L/gAVf3SnTEN8D2Q4i3s1jWoWxxz7jqFP543nJSkaCcf0gm3wnWfQ0Qc/Od8+PQ+KC+tcd7IlMQaw1KXpuWyOauA1COS2LankJU78xp8za83ZFKpcP7RPekeH2mFgjGmVXgpFK4GxgEPqepmEemHM4GtKb2A4DaSNHdfQ64FPqzvCRG5XkQWiMiCzMx2lMK6xyi4fjYccxV88094/oeQtaH66dG9E9mcVUBukZND6b3FOwgP8/HYxaPx+4SZy9MbvPSstZkkRgcY3TuJwd3jrPnIGNMqmiwUVHWVqt6mqq+JSBIQp6oPe7h2fUNl6m0vEZHLgVTgbw3E8KyqpqpqapcuXTy8dCsKj4Zz/gGXvAI5W+GZH8Ci/4AqI1MSAFielktZRSUzlu7kh0d1o09yNOP6JzfYhFRZqcxam8kPBnbB7xOGdI9jY8Y+S5sRYhWVSm5VEkRjOigvo49miUi8iHQClgJTReQxD9dOA3oHbacAO+u5/qnAvcC5qlpS+/lDxlHnwE3fQkoqTP8FvHUlI5Odp5am5fD1+kyyC0r50dFOZenMET3Ykl3I6vS6NYBV6Xlk7SthwiCnABzcPY7Sikq2eBixZJrv2a82Me7hz62pznRoXpqPElQ1D/gxMFVVjwFO9XDefGCgiPQTkXDgUmB68AHusp7P4BQIh36u6fiecMU0OPWPsOYDEl48mfMSN7EsLYd3F+8kKTrASe4X/WnDuuET6m1CqhqKelJQoQA2AsmLjLxiduYUNevcT1ftorC0gpv/u5CCkvIWjsyYQ4OXQiFMRHrgrKvwflMHV3FHLN2KsxbDauBNVV0pIg+KyLnuYX8DYoG3RGSJiExv4HKHDp8Pxt8BP/8MwiJ5vPgPHLfpSb5etZWzR/YkPMz5yDvHRnB8A01Is9ZmMqJXAl3iIgA4smssfp/YL1gPfvHaYi56+jtKyw+sqS23qIwl23MYf2RnNmcVcO+7yxsdHWbM4cpLofAgzhf7RlWdLyL9gfVeLu4m1BukqgNU9SF3332qOt19fKqqdlPV0e7t3MaveAjpeTTc8BXre57HNfou83zX8Nv0O+CLh2DTbCgr4swRPdiUVcDa3fu/7KuGok4cvL/vJCLMT7/OMVZTaEJuYRnzt+xhR04R7y3ecUDnfrcxi0qF208dyB2nDuK9JTt53cNcEmMON146mt9S1ZGqepO7vUlVLwh9aIeBiFjyTnuci0v+wP/CzyHWXwFfPwr/ORcePoKLV9zIHWH/Y9nXH1TnVJq1LoNKhZMH11ygZ3D3ONbubngIa1tT1Vb5Zb01u4CfvzSfnMLSOs99td4ZxtspJpz/m7WBikrv8cxel0VsRBijeydyy8Qj+cHAztw/fSWrGhk2bMzhyEtHc4qIvCsiGSKyW0T+JyLtcj5BezS8ZwKrw0ew54Q/INd/CXdtgcvegGOvI7yigNvC3uHilTdS+lAKCx8cz/q37mdSzCZG94yucZ0h3eLYvqeIfe2wrbukvIIJj87in597qkA2ac76LHY00C/w7uIdfLY6g2lL6oxZ4Mu1GSRGB/h/5w1nS3Yh7y+re0x9VJWv1mVywoBkAn4ffp/w+CWjSYoOcNHT33L/tBVszDzsFhg0pl5emo+m4nQQ98SZZzDD3Wc8iAr389VvJ3LTyQOcHZEJMHgynP4Q3PAV006bw89Lf83/5DS6hhXym8BbPF/xe/yP9HUmxX31KGyazYjEEkBZF9TUpKp8uzGL4rKKNnlvVWYuT2drdiHPzN5EZv7BDSBbvzufn70wlz/PXF3v83PWZwFO4RCsstL5Yj95UBfOGN6dgV1jmfLlBio91Ba2ZBeyI6eIHwza32TXOTaC1647ntOHd+e1eduZ9PfZXPH83AbTmKtqszu4G6OqbMjIZ+6m7Ba/tjH18ZIltYuqBhcCL4rIHaEK6HCUFBPe4HPnjxvGxNGDSIhyUm5TuAe2fgObv4Ytc+CL/wfABGBRRCzl7w6CAaOhy1HM2pPEnbPL6NYjhacuT6VPcnSDrwOwO6+YMJ+QHBvRgu8OXvx2K93jI8nIL+aZ2Rv5/dk1s5lk5pfw6txtjO6TyHH9OtWbK6rKXz9aQ6XCV2szKauoJODf/7slv7iMxdtzSI4JZ8n2HLZkFdC3s7NS7IqduWTtK2Xi4K74fMItE4/kjjeW8Onq3Zw+rHuj8X+1zpkQedLAzjX29+8Sy2MXj+Z3Zx7Fa3O38dJ3W7nyhXnM+MV4eiZG1Tj28U/X8cQXG3jrxnGM7dup6Q+tCUu25zBzeTqfrtpdnTzxg9vGM6xnwkFf25jGeKkpZInI5SLid2+XA/azpYWICInR4fvTYkd3cuY8nPkI3Pwt3LkJrniPytP/wmccR3FpOaz4H3x4JxPn/pwFkTfxyt6fkPmvU0h75UaY+6zTkb2v5vKhhaXlnPfkN5w35RtPwy0z8op5d3Fak7+0l2zPYen2HG6aMIAfHZ3Cy99vJSMoo2t5RSW3/HcRj3+2jitfmMfoBz/hqqnz6u0I/n5TNp+tzuDYvp3ILyln/pY9NZ6fu2kPFZXKvWcdhQg1mpC+XJOJyP5hvGeP7MERydE8+cWGJvs6vl6fyRHJ0RyRHFPv851jI/jFpIG8fv3xlJRXcuMrC2vUzt5bvIMnvnBmsn+4/OAXJJy7KZvzp3zD1G82k5IUxf3nDCUm3M+zX2066Gsb0xQvhcI1OMNRdwHpwIU4qS9Ma4hJhgET8Y27mde6/Zo74x+Bu7by9+HTuLz0HtLH3U9g2LlE+oW49dPgwzudjuxHB8Ij/eHfp8LrP2XdCzdwUcGr/CDvA95749+wYyHkptXJ1wSwcOsezvrXHH75xlI+WbW70fBe+nYLMeF+fjymF7dNOpLySuX/Zm2sfv7RT9Yxb8se/nrBCKZePZZLx/Zhc1YBd7yxhKeCjqusVP48czU9EiJ56vIxhPt9fLmm5tSVORuyiAr4OWtkD47r14lpS3ZUf+F/uTaDUSmJdHJrZWF+HzedPIDlO3J5+fut7C2o+z4BSssr+W5jNj+oVUuoz5FdY3n8ktEsS8vl3ndXoKos2LKH3769jOP7d+IHAzvz6epdB93hPmPZTqLD/cy/91RevvY4rj6xHz85rg/vL0tn+57Cpi9gzEFosvlIVbcBNYaKus1H/whVUKZ+Q7rH8eGKXazdvY//W1TIpWPPosfpIwAYUFbB799dzleLVvDAOB9ndsuFjNWwdzNlmes5Imsnvw64/RGb3FuVqE4Q2w1iu7K5JJbF2+HqqC5kxiYw77NNTE4+GWK6QHQyhO1vCsvML+GDZelcdmxv4iIDxEUGuHBMCq/O28aNJw9g5c5cnp69kZ8c14dLxvYBYOLgrvyhcii/enMJf/1oDRFhPq4Z348Zy3ayLC2XRy8aRXJsBMcPSObzNRnce9b+pqg5G7I4tl8nIsL8nD+6F3e/s5zlO3LplRjF0rQc7pg0qMbn9eMxKfx7zmbum7aS+6at5MiusYzt24kbT+5fXStYtG0vBaUV/GCgt/QpPxzajdsnDeSfn6+na3wEb87fTq+kKJ6+/Bg+WJ7Ove+uYH3GPgZ1izvAv66jslL5dNVuTh7UhcTo/Z/1NeP78eK3W3h+zmYeOHdYs65tjBfNXXntV1ih0OoGd4vjtXnbueONJcRFhvGb0wZXPxcZ8PO3i0Zxc2kFv5i7m6RrT2fc8U6ejbveXML7u9L5/I5xdJZcbn/uY5LZyx9P6UJ4cRbs201Z7i52pG3Bv28VPwvLJbzU/WWdgzPnvPqFEpwCIqYLeQWR3C8+zvCPgLnfQkwyvx4Yz7LF23j03W/4bEsZw3rGc1+tPga/T/j7RaMoKavkwfdX4RP495zNHNUjvjoNyKQhXbl/+ko2ZxXQr3MM6blFbMjYxyWpTuaUM0b04L5pK3l38Q5GpiSgChOH1PxiDw/z8f4vxrN0ew4Ltu5l4da9TFuyg49WpPPMFakc268TX6/PxO8TThiQ7PnvcPukgazcmctTszaSEBXg+StTSYwOZ9KQbtzLCj5bvbvZhcLStBx255Vw2rBuNfb3SIjivNG9eH3+Nm6bNLC6RnSwKiuVCtUafTftnary6CdrKSipYFC3OAZ3j6V3p2g2ZRaweFsOi7ftZVdeMWeN6MHFqb0b7dMzdTW3ULB1IdvA4O7xAKxOz+P/nT+8zj92EeFvF41i3ZNzuPXVRbx/23gy80t4Z9EObjx5AL27JAKJXH3xBVz23PdEZfTjtklH8vyczby4dgv5JeXcMnEAvzp1EJTto3DPTm569hPGdavkxrHxUJAFBZlQkEllQRbs2cA54XnEL/iCqlyHXYEPA8AWKMOPlHUn7KVeEN8D4npW34fF9+CJ07vxi9ICHpjhrEH98rUj8Pucf1qnuIXCF2syuHZ8v+pRR+PdZp6EqAATh3RhxtJ0MvJL6BwbzvB6OmEjA36O65/Mcf2dL/0tWQVc8+J8Lv/3XP564Qi+Xp/FmD6JxEV6yQbv8PmExy4ZzQPTV3Lp2D707xILQPeESEamJPDZqt3cPOHIJq5Sv09W7cbvE04Z3K3Oc9ef1J+3F6bx8ndbuf3Ugc26frD84jJueHkhK3fmcf1J/bnyhL7ERrT/FXo/WrGLKV9uJDzMV+/M9X6dY4iPCvCXD9fw2KfrOG90T64d3786XUx79czsjWzJLuT+c4Y2Ohgj1Jr7L8Dm/7eBIe4/6qE94vnJsX3qPSY2Ioxnrkjl/CnfcNMriwj4heSYcG6ZOKD6mHEDkrni+COY+u1m3pi/jYLSCs4c0Z1fnDKQo3o4BQ/+eKJ7xjN8HPx11kZOv2gC/Trv74iduWwnt65ZzPNXpjJpcGdn1FShU2jsydjBm7MXMbmP0jc8D/J2wu6VsP4zKNuf1C8cpxJSEB1LXlhnenzfH1Y6BUfv6GRuTtpFzuJ10Ps4Nq1IZ2hMGUM6idOBLsL5o3vx8crdzFyezo+PTsHna/q3St/OMbxz8wnc+MpCfvnGUgB+9cNBTZxVV3xkgMcuHl1n/6Qh3fjH5+vI2ldC52aM8vpk5S6O79+JhOi6hdSgbnFMGtKVl77bwvUn9W/22t8AewpKuWrqPFbuzGNs3yT+9vFanp+zmRtO6s+ZI3oQGfATGfARFfAT1o5qESXlFfzlwzUM7hbH+7eNZ1duMet257M1u5B+nWMY1Xt/v9KaXXn857utvLtoB9OW7OSdm09ot6O31u/O55GP11JRqazdlcdzP0tt8VGCXklDnWIikk/9X/4CRKlqm/ykSE1N1QULFrTFS7cL//luCycMSObIro3/6vloxS5ufGUhAA/9aDg/Pa7mSqcFJeX87IV5dE+I5LZTBjb4Kyojr5jxf/2Si8em8Kfznf6LNbvy+PlLC/CJ8OVvJlT/uvekOA/y052CIvg+f9f+x/t2gzaSu8gXBpEJVEYksHKvsLcimiP79KJn9+4QmegsfBQRB+ExEB7r3CJia2yX+qO4d8YG3lmyk+m3nthiXxYrd+Zy1hNzeOTCkVyc2rvpE4JsyNjHqY/N5o/nDuPKE/rWe8y8zXu4+JnvuGvyEH48phfhfh/hYT6iw/37R7A1IT23iMv/PZe0vUX830/HMOmobizZnsPjn65j9rq665WcOaI7vz9raJ1huAdjX0k5j32yjguPSWFoz3jP5z0zeyN/+XANL197rOd+oIz8Ys791zeEh/mY8YvxJER5rxW2lqumzmPh1r3ce+ZR3D99Jd0TInnx6mNr/BA7WCKyUFVTmzzuUEv61dELhQPx9OyNLNiyh6cvP+agfu3d9fYy3luyg2/vPoU5G7K463/LiI8M8MwVx3B0n6QWjNhVWQHFuSxdv5Xfvz6HK49O5Isl67jmmE6kdvdBUQ4U50JxDmu3pFGYt4eRnRV/SR4U50BF/SONalPxQSAGqSpAImIhEA3+cAiLCLqPcDrY69zX3heB+gPc+c4aenVO5JdnjHD2+wMNX88fAPfL/KlZG/nrR2v49u5TGvwCVlUueOpbFm3LqbG/S1wEJw5I5oQjO3PikZ3p1cD5a3blce2LC8gtKuPfV6ZyfP+afSlLt+ewbnc+xWUVFJdVsiuvmFe+34rfJ9w+aSDXjO/nuf+hslIRod7C6skv1vPoJ+uICvj520UjOXtkzyavl7WvhIl/m8XYfp144aqxnmKosnDrXi555jsmDO7Ks1cc46lW6UV6bhEFJRUc2TW22deYvS6TK1+Yx71nHsV1J/Vn4da9XPefBagqz/0sldQWmPcCViiYFrR+dz4/fPwrhrgrwI3tm8SUn46ha1xkSF+3vKKSY/70GSXlzhfUd/ecQo+Eml92uYVlbMrat79wUnUKhdICKMl37kv3ObeSfQ1su8eV7IOyQicPVUWJM1y3wr2VlwTdt/CyH26Bkl+qVOAjMSYKfH4Qv3Nf/TgMfD5KK4X8UqjER4V7yysVMouUfeU+SgkjOT6WYb270Ck+FsLCqfCFM3/bPr7enIc/EMGl446kZ6d4p5DyhblzWnT/Zxj0OLughGmL01iVnkfXuHD6JEURHe4nJtxPIMxPQbmf/HIfeWV+9pb6yCpWdhcKGUXK2AE9eOBHRyOBKLdQjKSgws/4R75kcPc4yiqUhVv3cvOEAfz6tMGN1jrvfXc5b8zfzkd3nNSsL+Gp32zmjzNWcdfkIdw0YUDTJzRhc1YBFz39LVn7SjnmiCQuO7YPZ43ocUDNeuUVlZzxz68prajkk1+eRESYc+6WrAKufnE+O3OK+OelRzN5eOMTML2wQsG0qGtenM8XazK46oS+3HvWUa02WuX21xczbclOBnSJ4fNfT2iV12ySKlSU1So4qh6XsGjTbv4yYyn3nt7fyWFVVaDUV7i45xQWFvLW/C2M6R3PiB6xTm1JK5z7yvKgx7X3Vzr3FWVoRSmlJcUUFRVRVFxImJYR468kQsqhohQ/7WflvhINIyw8Ep8/jIIypagc/P4wIiMChAcChPmDCkXxU1wJGzKLSIqNpFdSrLu/9r/BWgVKrRqK4jTRZReUMrB7Ap3iYhB/wCkU/QHwBdz7MLcWWPu5sOpjcosreOGbrZRUKMf178LitFwy9pURHgijX+c4usRH0SU+ku4J0fROjsXn8+2Pt6IcKsugooy5G3fx0dLtXDKmB0O6RLh/y1KoKKOopIRZq9LI2VfI2N5xHJkcAUdfDv0nNOsz91ootP+hBqZd+NuFI9mUVdAiKRwOxClDujJtyU7P7cetQsRp/gkLh3r6Aod1Hsqqj8p5c28vRk8Y4emS73y/lfvLV/DJ+SdBM4ezCk44EYCvuIznvtrE83M2U1haQefYCP503lFMHtKpbiFVWeF+gUrNL9Lqx7L/+er9QklFJSWlFUT7ywnTMigvdgq58mKoKKW8tIhHZy6jorSY35xyBBFSTllJEVO/Wkv3GDh3WGeoLCemsoKdu3JZnraHyvxyfFJJXMBHXKSPivJyKirKKSsvJ+CLZnC3RPCpE3Nwl2dTP25VEZzUJUWlOWzatYfdmdl0ihISIoRwqaj+spbKMnyVZTW+vNGa+cUSgF9WbWyEiQBVXRVZ7s2D44DjAsDyoJ3iB3+AKH84k31h5IbDvh0+9mZHknDkaZ5mHB8MKxSMJ8mxEW0yGmLikK6M7ZvEj8f0avXXbq6IMD8nDerCjKU7yd5XQsDtDE5JjOK0Yd0Z1jO+Tjv7J6t20zc5moEH0TYdLD4ywK9PG8yVJ/Rl9tpMThnSdf8Q5kDLdBhXFUANCQNOjT2JC5/+Dsnp7+SQ+m4Lfy5cyWs/PR7cuSECDAIGVCprduUxb/MePtu8h525xSRGBUiMDpAYFeDsUT0JHOSPEj/Qr7iMj1bs4oPl6cxZn0V5PalchnSP45xRPd10KTFQWUlxaQnpe/L53duL2JCRz1M/OZrUIxKd2lqtW3l5Bem5hcxZl8HzX2+kT1IkD547lJTESNQXxvfb8pgyextrM4v5z89P5KiU5P01Et/+r30B4iqVR6ev4JXvt3FPzhBuOKhPoGnWfGRMCHy/KZu/fLiGkrIKSssrKSl3Om0rKpVeiVFMHt6dmHA/a3fns273PjZnFXD9Sc4X5+HmrreX8faiNN67+URueHkBPROjeOvGcZ5HS4VSbmEZX6zdzd6CMqdTHCgsq+Dz1Rks3LoXgP6dY8grLidrn9OX5PcJz15xDJOOqjuXpD7fbsjillcXUVGp3Dl5CDOW7mTe5j306RTN784cwuThPZq8hqryxvztnDWyxwHNqQlmfQrGtDN7Ckr5bPVuPlqxy/2FWkm/zjEM7h7HoG5xXH78Ec2a29De7S0o5ZS/z6KiUskrLufFq8cyodYiUu1R2t5CPliWzvwte+kcG07PxCh6JEQyMiXxgCfCbd9TyHX/WcCaXfl0jo3g9klHcsnYPtXL87YGKxSMaceKSisQoU1nrramNxds57dvL2NErwSm33piu6gltLaCknJmrc1k4pAuRIe3fsu9dTQb044dzGzkQ9GFY1LIzC9h4uCuHbJAAIiJCOOskU03FbU1KxSMMSFXtfCRaf/aT1ITY4wxbc4KBWOMMdWsUDDGGFPNCgVjjDHVQlooiMhkEVkrIhtE5O56nj9JRBaJSLmIXBjKWIwxxjQtZIWCiPiBKcAZwFDgMhEZWuuwbcBVwKuhisMYY4x3oRySeiywQVU3AYjI68B5wKqqA1R1i/tc+0nfaIwxHVgom496AduDttPcfQdMRK4XkQUisiAzs+7KUMYYY1pGKGsK9U1bbFZODVV9FngWQEQyRWRrM2PqjOektq3K4jowFteBa6+xWVwH5mDiOqLpQ0JbKKQBwYvUpgA7D/aiqtrsxPoissBL7o/WZnEdGIvrwLXX2CyuA9MacYWy+Wg+MFBE+olIOHApMD2Er2eMMeYghaxQUNVy4FbgY2A18KaqrhSRB0XkXAARGSsiacBFwDMisjJU8RhjjGlaSBPiqepMYGatffcFPZ6P06zUWp5txdc6EBbXgbG4Dlx7jc3iOjAhj+uQW0/BGGNM6FiaC2OMMdWsUDDGGFOtwxQKTeVhCvFr9xaRL0VktYisFJHb3f0PiMgOEVni3s4MOuceN9a1InJ6CGPbIiLL3ddf4O7rJCKfish69z7J3S8i8oQb1zIRGROimAYHfSZLRCRPRO5oi89LRF4QkQwRWRG074A/HxG50j1+vYhcGaK4/iYia9zXfldEEt39fUWkKOhzezronGPcv/8GN/aDWhatgbgO+O/W0v9fG4jrjaCYtojIEnd/a35eDX03tN2/MVU97G+AH9gI9AfCgaXA0FZ8/R7AGPdxHLAOJx/UA8Bv6jl+qBtjBNDPjd0foti2AJ1r7XsEuNt9fDfwV/fxmcCHOBMTjwfmttLfbhfOxJtW/7yAk4AxwIrmfj5AJ2CTe5/kPk4KQVynAWHu478GxdU3+Lha15kHjHNj/hA4IwRxHdDfLRT/X+uLq9bzfwfua4PPq6Hvhjb7N9ZRagrVeZhUtQ8h7ZsAAAaCSURBVBSoysPUKlQ1XVUXuY/zcYboNpby4zzg9f/f3tnFSlWdYfh5DaIpCq2mNGj8rYdWkoZDSwipWi9arBqFatMGYoK2NbYRTbWmMYSk6UUvuGjai9ZgIdiKCNEWW71oAoYL0Aa1chSktCA/9UIOEMEfqBRbeHux1szZZzpzYGDPnpb5nmQye1bW3uudb/98e6299vfZPmp7N7CD9B+qYhbweF5+HPhaoXyZEy8BH5fU6aSzXwZ22h7pLfaO2cv2euBgk/basc9XgedtH7T9LvA8cGPZumyvcZoKDvASJ5jZl7WNtb3B6cqyrPBfStM1Aq32W+nn60i68t3+N4GVI22jQ/ZqdW3o2jHWK06htDhMp4uky4EpwMu56L7cDXys1kWkWr0G1kjaKOmeXPYp24OQDlpgfBd01ZjN8JO12/aC9u3TDbt9m3RHWeMKSa9JWifpulx2cdZSha529lvV9roO2Gf7zUJZ5fZquDZ07RjrFadQWhym0xIhnQesAh6w/QGwCPg00A8MkrqwUK3ea2x/nhTifJ6kL41Qt1I7Kr0JPxP4bS76X7DXSLTSUbXdFgD/Bp7MRYPApbanAD8AVkgaW6Gudvdb1ftzDsNvPCq3V5NrQ8uqLTSUpq1XnEJH4jC1g6SzSTv9SdvPANjeZ/uY7ePAEoaGPCrTa3tP/t4P/D5r2FcbFsrf+6vWlbkJGLC9L2vsur0y7dqnMn35AeMtwB15iIM8PHMgL28kjddPzLqKQ0wd0XUK+61Ke40CbgeeKuit1F7Nrg108RjrFafQ1ThMecxyKfBX2z8rlBfH428DajMjngNmSzpH0hVAH+kBV9m6xkg6v7ZMelC5Jbdfm71wJ/BsQdfcPANiOvB+rYvbIYbdwXXbXgXatc9q4AZJn8hDJzfkslKRdCPwMDDT9oeF8k8qJb1C0pUk++zK2g5Jmp6P0bmF/1Kmrnb3W5Xn61eAv9muDwtVaa9W1wa6eYydzpPz/6cP6an9dpLXX1Bx29eSunKbgdfz52bgCeCNXP4cMKGwzoKsdRunOcNhBF1XkmZ2bAL+UrMLcCGwFngzf1+Qy0XKprcz657aQZt9DDgAjCuUVW4vklMaBP5Fuhv7zqnYhzTGvyN/vtUhXTtI48q1Y+zRXPfref9uAgaAWwvbmUq6SO8EfkmOclCyrrb3W9nnazNdufw3wPca6lZpr1bXhq4dYxHmIgiCIKjTK8NHQRAEwUkQTiEIgiCoE04hCIIgqBNOIQiCIKgTTiEIgiCoE04hOCOQdKGGolru1fConKNPchu/lvSZE9SZJ+mOkjTPyvo2Sdoq6e5cfrukz5bRRhC0S0xJDc44JP0YOGz7pw3lIh3zx7sibLiWc4DdpHnme/Lvy2xvl7Qc+J3tP3RXZdCLRE8hOKORdJWkLUox8QeACZIWS3pVKX79jwp1X5TUL2mUpPckLcx38Rskjc91fiLpgUL9hZJeUYr9/8VcPkbSqrzuytxWf4O0caQXkQ5CPbTC9hx87Wbg57kXcbmkPkmrlYIWrpc0MbezXNIiSS9I2i7ppg6bM+gBwikEvcAkYKntKbbfJsWpnwpMBmZImtRknXHAOtuTgQ2kt0WbIdvTgB8CNQdzP7A3r7uQFPlyGE6xplYDb0laIWmOpLNsvwD8EXjQdr/tv5OStd9r+wvAfNKbtDUuAa4HbgUW5x5HEJwy4RSCXmCn7T8Xfs+RNEDqOVxNchqNHLFdCz29kZR4pRnPNKlzLSkHALZrIUT+C9t3ATOAV0mJVBY31lHKnjYdWKWUGewR4KJCladtH7e9jRTioq+FziA4KUZ1W0AQVMA/aguS+oDvA9Nsv5fH789tss5HheVjtD5Xjjapc9IpGm1vBjZLWkFKsHJ3QxUB79huHH6qb+IEv4OgLaKnEPQaY4FDwAcaylhVNi+SMnkh6XM06YlIGqvhuSv6gVp2uUOk1Iw4ZdEalHRbXu8sSZML630jR8ycSBpKKiaKCYK2iZ5C0GsMAFtJkS53AX/qQBu/AJZJ2pzb2wK831BHwHxJS4AjwGGGnlusBH4l6SFSGsbZwKI8q2o0sJwUwRNSRMz1pMxc9zilrwyCUyampAZBySglbhll+595uGoN0Oeh/MlltRNTV4PSiZ5CEJTPecDa7BwEfLdshxAEnSJ6CkEQBEGdeNAcBEEQ1AmnEARBENQJpxAEQRDUCacQBEEQ1AmnEARBENT5D+syozD7AnyBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title('Loss versus training step.')\n",
    "plt.xlabel('Training Step')\n",
    "plt.ylabel('Loss (sparse softmax cross entropy)')\n",
    "plt.plot(tr[:,0],tr[:,1],label='Train')\n",
    "plt.plot(ev[:,0],ev[:,1],label='Eval')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl8VNXZwPHfk30lkIQdwq6CqIC4INalVkXrVrV17atW61Kt1q3Vt621trZqa/u6tRX3pS641J2iVVFxK4ssyiIICEG2JECAZLI+7x/3TjKZzEzuhNyZhDzfz2c+mXvnLs/cJPPMOeeec0RVMcYYYwBSkh2AMcaYzsOSgjHGmCaWFIwxxjSxpGCMMaaJJQVjjDFNLCkYY4xpYknBmC5ORHaIyPCO3tZ0T5YUuhERmSkiW0QkM9mxGBCRI0SkdFePo6p5qrqyo7ftCCIyVERURNISdU6zazwlBRHJFpE9/Q7G+EdEhgLfAhQ4KcHn7nIfCJ0l5s4Sh+k+2kwKInIiMB/4t7s8TkRe8Tsw0+H+B/gEeBQ4L/QFN+nfKSJfi8g2EZklItnua4eKyEcislVE1orI+e76mSJyUcgxzheRWSHLKiKXi8hyYLm77i73GJUiMldEvhWyfaqI/K+IfCUi293XB4vIfSJyZ1i8r4rIz8LfoIj8Q0T+HLbuZRG5xn0+QEReEJHNIrJKRK4M2e5mEXleRJ4UkUrgfBE5UETmuPFuFJG/uNu2+oYvIqtF5Dvu84j7hW2fC0wHBrhVOjvc+KLF8bH7O1gvIveKSEbYtR7pPn/UvWavu9fxUxEZ0c5tjxGRZe7fxN9E5L3Q33nY+4n2nt93f2513+Mkd/sficgSt+Q6Q0SGhMV4pYisFJEyEfmTiFitRqKoaswHMBcoAD4LWbewrf3s0bkewArgJ8D+QB3QN+S1+4CZwEAgFTgEyARKgO3AWUA6UASMc/eZCVwUcozzgVkhywq8BRQC2e66c91jpAHXAhuALPe164FFwJ6AAPu52x4IfAOkuNsVA1Wh8Yec8zBgLSDuci+gGhiA8wVoLnATkAEMB1YCx7rb3uxel1PcbbOBj4Efuq/nAQe7z48ASsPOvRr4jvs84n4R4o10nEhx7A8c7F63ocAS4Gdh13qk+/xRoMK9bmnAP4Fn4t3Wvc6VwKnua1e5cV0U5b1Eu1ZD3XOmhWx7Cs7f42j32L8CPgqL8V2cv50S4Mto57VHxz+8ZN96Vd3mYTvTSYnIocAQYJqqzgW+As52X0sBfgRcparrVLVBVT9S1RrgHOA/qvq0qtaparmqzo/j1H9U1QpVrQZQ1SfdY9Sr6p04iSdYLXkR8CtVXaaOBe62/wW2AUe5250JzFTVjRHO9wHOB0qwBHI68LGqfgMcAPRW1VtUtVadevUH3OMFfayqL6lqoxtzHTBSRIpVdYeqfuLxfbd3v4hxqOpcVf3EvW6rgfuBw2Ps/6Kq/ldV63E+6Me1Y9vjgS9U9UX3tbtxkng08bznS3D+Npa4x/4DMC60tADc7v7trAH+D+eLiUkAL0nhcxE5G0gVkVEicg/wkc9xmY51HvCmqpa5y0/RXIVUDGThJIpwg6Os92pt6IKIXOtWGWwTka04JdBiD+d6DKeUgfvziUgbqaoCz9D8AXI2zgcdOElxgFsFs9U9//8CfaPFC1wI7AEsFZHZInJC9LfaIftFjENE9hCR10Rkg1ul9Aear1skoR/eVTjf3OPddkBoHO61jdUoHs97HgLcFfJ7qMApHQ4M2Sb0GnztxmMSwEtS+CmwN1CD82GyDWhVn2s6J3HaBn4AHO5+qGwArgb2E5H9gDIgAIyIsPvaKOsBdgI5Icv9ImzTNASv237wCzeWXqraE+dvSTyc60ngZDfe0cBLUbYDeBo43f3WeRDwQsjxV6lqz5BHvqoeHyleAFVdrqpnAX2A24Hn3baAFu9dRFKB3h72CxdtiOLw9X8HlgKjVLUHTjKTVnt1rPXAoOCCiEjocrgY7znSe1wLXBL2u8hW1dAvm4NDnpfgVCGaBPCSFPYHblLVA9zHr4AxPsdlOs4pQAPO72yc+xiNU9XyP6raCDwM/MVt6EwVkUni3Lb6T+A7IvIDEUkTkSIRCVYvzAdOFZEct+HywjbiyAfqgc1AmojcBPQIef1B4HduaVREZF8RKQJQ1VJgNk4J4YVgdVQkqvqZe44HgRmqutV96b9ApYj8QpyG9VQRGSsiB0Q7loicKyK93WsUPE4DTh13loh8V0TScerEMz3sF24jUCQiBdFicOXj1O/vEJG9gMva2L4jvA7sIyKniHMH1OVETvxAzPe8GWjEacMJ+gdwo4js7e5bICLfDzvk9SLSS0QG47RnPNsh78q0yUtSmAG8IyKhxewHfYrHdLzzgEdUdY2qbgg+gHuBc9x/+OtwGnln4xTlb8dp2F2DU7d8rbt+Pk4DMMBfgVqcD7bHaK6miWYGzt02X+JUBwRoWUXwF2Aa8CbOB+BDOI2sQY8B+xCl6ijM08B3cEq2AKhqA3AiTlJchVNCehCnCiuaKcAXIrIDuAs4U1UDbhvbT9z91+GUHErb2i/84Kq61I11pVuVEq2K5DqcqrDtOO0gvn9AulWN3wfuAMpxvlTMwakxiCTataoCbgU+dN/jwar6L5y/sWfc6rDPgePCjvcyzo0B83ES1EPglDjdcxifBO/SiL6ByGfAr4E/AReq6kci8pmqjk9EgMYAiMhhONVIQ91voyaB3BsSSoFzVPVdn8+lOFVlK/w8j4nMS8cYVdXXRGQZ8KyIPEz0ulBjOpxbRXMV8KAlhMQRkWOBT3Fu670epx0j3jupTBfjpfpIwGlIwrnV7zBgXz+DMiZIREbj1FH3x7k10STOJJw7wspwqt5OidWeY3YPbVYfRdxJpMStbzbGGLMbiVp9JCI/V9U7ROTuKJtcGWW9McaYLipWm8IS9+fcRATiVXFxsQ4dOjTZYRhjTJcyd+7cMlXt3dZ2cVUfuXcg5Klq5a4EtysmTpyoc+bMSdbpjTGmSxKRuao6sa3tvIyS+pSI9HB7Jy4GlonI9R0RpDHGmM7Fy91HY9ySwSnAGzhdzn/oa1TGGGOSwktSSHfvEz8FeFlV67B+CsYYs1vykhTuxxkrPhd43x1oLGltCsYYY/zTZo9mVb0bZyz1oK9F5Ej/QjLGGJMsbSYFd7TM03BmUArd/pY29nsYOAHYpKpjI7wuOANnHY8zjvv5qjrPc+TGGGM6nJfqo5eBk3GGPd4Z8mjLozgjJ0ZzHDDKfVyMM2a8McaYJPIyIN4gVY314R6Rqr4vIkNjbHIy8Lg7o9MnItJTRPqr6vp4z+W3QF0Dz80t5fv7DyIrPbXV60vWV7KxMsARe/aJuP+0OWsprajyfL6xAws4Zu/IQ9f/67NSVm32kpMde/brwXf37R/xtRfnlbK6zPuxwh0yspiDhxe1Wt/QqDwzew0n7DOAgpz0Vq+v3LyDl+Z/Ax77yKSnpnDuwUPolZvR6rW1FVUsKN3KCfvGPzHXjpp6Hv1wFbX10cfYmzCkV8Tfq6oybc5avr1XX3rnZ7Z6vXRLFc/NKaU9w8jsrkSEUycMZEhR6/mGynfU8OQna2ho9DjeoQinjBvA8N6xJpVzrNi0g1cWeP9768yOGt2X/Qb39PUcXpLCRyKyj6ou6uBzD6TlePql7rpWSUFELsYpTVBSUtLBYbTtoVmr+NOMZVTsqOWq74xq8VpdQyM/+ec8Nm+v4bObjiY9tWXha93Wan7+/EIAxMNcWapQkJ3enBTqqiElDVLT2VlTz9XPLojrWCkCe/Q9jFF981u89tmaLVwzzfuxIh37iU++5r2fH0mPrJYf/C/MLeWX//qcJesr+f0p+4Ttp1z33ALmrdnq+byqkJIiXH7kyFav3fPOcqbNKWVAz2wmlPSCugCgkJ7d+kAR9r3/vZVR41CF7PRU3v/5ka0++Gd8sZFfvLCIE/cr556zxkNjI9RUQloWpGXy65c+591lm9t1beORSgM51FBDOrW0TsCdiSp8uqqcp398MBJ2YW6bvpTn5pbG9Tfx0Yoynrt0UqtjtdxOuXbafBaUbms6dga1ZFBPFVk0eqos6Tz69MhKXlIQkUU4t56mAReIyEqcCTYEZzjtXR0pNdJvMmIqV9WpwFRwejTv4nnjsmVnLf+Y+RUpAg98sJJzDy6hKK/5A+K5OaWscr9tLyzdyv5DClvs/+FyZ1rkN68+jD2CH8wN9VBVBjs2wo5N7k/n+VerVlK2YQ0Nd91AatVm54NGUqHnYDS3hFvSMtl//AT2HjseCodDzyGQnhU19sPueJc/v7mM+3/Y3JFRVbn930spzsvgveuPJDfTy3eDlj4v3cqp987kybfn8ZPJ/aF2J9TupLaqkjkzPuXk1B2Uz57DhpK19OtdDBm5kJHLe6uq+WLNJm773gTOPGhI2ycCjv/ru3z+5XIYXd3imun2jRy9eBHfS6+g72Pb0fRKJLDN2SmzB+T1gby+YT/7QV5fyujJax8u57Txe3DnGftHPO/qsp2c9JcZPDX9Xa46qKDp3I3bN9D46UIeTi+j95It1N1RTXp1GagzuVqjpPF/jRmkFuSTl1/gvve8pmvQejkP0nOgobbpOlK7I8rPsOf1IXP3ZPUMe78hz/P7Nq/LLoQUjx+Gqs4Xk2gxNNQ6sUd7j2nNpbvHPlrNb175gg+Wl3HYHs2jLSzfuJ0X5pXy428N45ff9Tap41OfruF//7WId5Zu4qjRfaGxAarKW/wvsX0DX3+9ios2fslBA+rpI9uc9TXbmg+Ult0cb2Z+7N9Rq+fhP91H6i4k5/rasOscdt0H1LX/2B5FHebCvfU0KlX9us2DO9VHr0VpaL4fmKmqT7vLy4Aj2qo+SvQwF398YwlTP1jJfWdP4Iqn5nHhpEH88juDoXYngZ2VXPbI+/TLaqBsSwWn7d2LKXvktfhlfrD4a7Zv28Jxw9ORHZtgxwbYWUbE/JdZwI6MQj7flsVeI0fSs/cgyOvtfPutWMm2dcug4isKJLQqSqBgMBQOc5JE6KPXUO75YB13vrWMly6ewLg+6VC7g7nLS7n1pTlcNqkvR4/Mi/APH+NDqHYn1G53fjbWt/u6qqQg0T4o03Oc4wc/hHdsJiXCbJaN6bl8XZNHVUYxq2tyGTd6TwYOGuJck52bw5LuJifBRoojp7j5AzQjB3aE7FvXunqtUVLZ1NiDjJ79+WJbJo25fTl8wljI7oXW1/Dip8uor97Oafv0Iq2+KsZ13AHRpocI/bCKdJ0yQz6Q0nOgvrr5vW7f2PzhWBeh2lJS3WThJozg9Y4W4650S0pJb4pTM3JZUt5AXWoO+w4fgGTkgqTy6apyNm+v4egx/chM85asGrWR2UtWUqRbGZGzE9m5OeK13Ek2W1J6MXDQUCQ/JFGmZkBtlft+o/2tR0m+bUnNjJEkM51jhR67JiSGxjY+9L97JxxwkfdYQngd5sLLzGsHA1+o6nZ3OR+nl/OnHoIYSvSk8F3gCpy7jw4C7lbVA9s6ZiKTwvpt1Rzxp5mcu5fw603XUL99I2nq/YNQ07LZUp+OpudS1GdAhG+tYd/m0rNZ/E0lx9/9AfedPaFVW8BDs1bxu9e+YP51E+kZKIWKlc2P8q+cn9UVLWPIyEVrq0jx/I8tET6oI39jqqjP4N5Z69lv+EBOPnAUVZLFT5//kgF9e/O70w7g0Q+W8ursFdx5ykiG5jUyZ/laXpm9nLPHF7FXr5TYySgjt+narK7J46EF1Zz97QMYPWqkc71y+/DEZ+X8+qXPefPqw7jwsdn0yErn1SsOJSUlSnVCbRXs3MS60tX8/pmZnDA8he8OS22ZPOqqILd30+9me3ohv3+vnGFDhnPpCYcQyCzmqL8tpLhHNi9dPpkHP1jFrW8s4ekfH8ykEUX8Z/FGLnp8Dn88dR/OOrCNqs7Qb+F1O0M+THIhpXXbVbvU7IhQIt3Ycl1tlZtk2ijNRHotNdO5ZuFfGKL8bjeUlbNmw2b27JVCQWoNNXUNrN8WoGdOOj2z4/uGvVVzmVOezsjhwxk6ZHjI/5Pzd/PCslqufXkF9/9wf46N0kbnWUO98zsKfojXBT/MvX6hcn/WVTtfPGJe2/zov4P8vk6Jph28JgUv9QZ/ByaELO+MsC5SAE8DRwDFIlIK/AacSk9V/QfOkBnHAytwbkm9wEMsCXX328tpVOXS0bWwYh01e5/F3QsbGT6wD8eMG8Fv/r2aAb2Lufq743l0bhlPzi3nlWuOISfPqTJYunEnx931AX8+cT9O33+Qp3MOLnTqwtdEaJheW1FFfmY6BUV9QfrBoAi/3+otULHKTRarkOoKFm6sY/qX2zlt0p7saMzi/k82cu5hozl0zNDWf3xp2Z6rFgqB6qpFXDd3LRNOOYJpc9bydrXy2omHQu8CvnfcMP666F1uXtyT+3+4P1e9+h6F/cbz29Mmx9WQ0bumnqfnv0lu7XBGl+zVtP7D5csY2DObUX3yuPboPfnZs/N5fdF6TtwvSqNzRg5kDOUP0yt4L/VgfnfGkZDXupE4VD7Qq3Ypt7//FYc3DubDRWWsq6zlTz8Yj4jww0lDePjDVdz+76W8cNkh/GnGMoYV5/J9L79vETemHKDNwSvbJ9MtVRSN8Of4cerdqJx31wfU1Dfw1pWHc95Dn7KiegfvXX0kxFmN2aNR+eu9s9i2qY53fngEGSGljEBdA3+eOZNxg3tyzJi+MY7iUWoapBZAVqwpvXcPXn4LoiHFCVVtdCd7j0lVz2rjdQUu93D+pPhq8w6mzSnlhwcPoXfKfAByv3MD1TlVXDNrFVPy+/FGoA9vfO9bMKAHo+rLWDH7Uz4ty+TI4h4AfLjCaU+YPLL1HTrR5GelU5ibETEprKmoYnBhTsyGNbJ7wcBeMLA5Z+9V38Dld77HByudxurM3mOZdMy3nFboXXTVUaN4cV4pv3nlCz5ZWc4J+/Zn7EDnH6cgO52fHDGCP05fyrXTFrBuazW3nbZP7PgjyM1MY0JJr6brCc4dTh99VcaUsf0QEU7abwD/eO8r7nxzGVPG9mvV4B+0qHQbry9cz5XfHklxGwkh6LLDR/DUp1/zu9cWs2R9Jd8aVcwhI4sByEpP5WffGcUvXljEVc98xrKN27n37PGkRTl/d5eaIlx/7J5c9Pgcrn52Pp+srODmE8e0q10rJUX4+ZS9OO/h//L0f9dw3iFDm1578pOvWb8twJ0/2C/uv7fuzstf7koRuVJE0t3HVcBKvwNLtr+8+SWZaSlc8e2RTgMWQE4Rlx0xkpyMNN5YtIGTxw1gzAAnAew/pBeZaSnMCvngmrWijBG9c+lf0PadMKEGF+awNkpSKCnMifu9ZKalcs3Re/DFN5WsLq/i+mP3JLUDEgJAv4Iszp88lHeWbqKmvpFrj9mzxevnHTKUvj0yeW3heg4ZUcSh7odpvCaPLObzb7axZWctAJ+v20ZloJ7J7vFS3A+b1eVV3D59Ka8s+Cbi43evLaZXTjoXHTbc87kLctK59IgRfPRVOVuq6rj+2Jbv8bQJgxjeO5fXFq5n7MAeHD828i3AxnHU6D7sP6QXry1cz6Be2Zx1UPvvKDxsVDEHDy/knneW8/L8dbyy4Btenr+O+95d4STvEe37e+vOvKTnS3GGufgVTovT28CP/Qwq2bYH6nh90XouOXy4822yqtxpmMrIozBT+MmRI7j77eVcc/QeTftkpadywNDCpm+ztfWNfLqygh9M9FZtFKqkMIcFa7e2WNfYqKytqOLbe0XuC9GWk8cN5KFZq8jLTOOo0e07RjSXHT6C5+aU8t19+jOsuOU96FnpqVx3zJ7c+OIifj5lr3Z/azt0VBF//Q98vLKc4/fp35R8Q//pv71XHw4aVsiDs1bFPNZNJ4xpdRttWy44ZBj//GQN+w/pxb6DWt4SmJaawi+m7MVlT87lhimjo7dpGMDpr3DDcXtxxv0fc/2xe5KZ1v72ExHhF1P24rS/f8RVz8xvWp+aIvz82L1i7Gmi8ZIURqnqmaErRGQysNmfkJJv/TbnToO9B7j1h1VlkFPUVA9+2eEjOOfAIa06Zk0eWczt/17Kpu0BVm3eSXVdQ9M32XiUFGbzxqL11Dc0NlVDbN5RQ019I4PbUVIA55/k+UsPQYQOL073zMlg5vVHkBOhYx/A9ycO5ugxfemZ07rzmVf7DupJXmYas1aUcfw+/flwRRl79ctv0X9ARHjsRwdSuiX63PJpKcKQovivYXZGKjOuPoyMKNVCx+7dj3m/PnqX3mN3csDQwg67XuNLevHRDUexo6b5JpC8zDT6FUS+VdvE5iUp3EPrRuVI63YbG9yk0K+H+0dVVeEkBZeIROype+jIYm4HPlpRzsrNO0gROHiE9/aEoJLCHBoalfXbAk1JINjG0J7qo6DsjA66oyWCtr557+o/f3pqCgcPd0pi1bUNzFm9hf+Z1Pqu6az0VEb2abuXa3vktVHvbQkhPh15vSwBdJxYndcmAYcAvUXkmpCXegD+fbp0Ahsqw5NCOeQUxtjDMWZAD3rmpDNrRRkrN+9gv8E9466mAFokgqbn5bueFLq6ySOL+c+STfzrs3XUNjQyeZTVFxvT0WI1NGcAeTiJIz/kUQmc7n9oybPRLSn06eFWTVSVtygpRJOaIhwyooiZyzaxoHRbuxtVS8JKB8HnIjCwZ3yN1ruT4PW8++3lpKcKBw5tO1EbY+ITtaSgqu8B74nIo156L+9O1lcGKMzNaB78zmNSAOfb7BuLNjQ9b4/+BdmkpUiLpLC2oooBBdkt7sXubkb2yaNPfiYbKgMcOKywXbcxGmNi8/JfVSUifwL2Bpoq7lT1275FlWQbtwXoG6w6aqiH6q2ek0Lw22x2eirjS9o3cFVqijCoV3arkkKwY1t3JSIcOrKYFz9b1+5SmDEmNi9fO/8JLAWGAb/FmZpzto8xJd2GygD9glVHga2Aek4KJYU5DC3KYdKIol261S68r0J7+yjsbg7f0+n5GzqgmjGm43gpKRSp6kMiclVIldJ7fgeWTBsrA+w7KHg7anPHNS9EhCcuPGiX7/QpKczh9UXO2IDVtQ1s2l5jSQE4cd8BDCvObdVXwBjTMbwkheCwfevdQey+AeLvkdVF1NY3Urajtrn6qCkpeG/UbG9fglAlhTlsrapjW3Udm9y7oTriuF1dSopYQjDGR16Swu9FpAC4Fqd/Qg/gal+jSqJN250P4P4F4UkhsXXYwVLB2ooqNrpJwUoKxhi/eRnY7jX36TbgSH/DSb5gx7XWJYX4O6HtisEhSWGDJQVjTIJEbWgWkSwROU9EThLHL0TkNRG5S0R221s/mjqutSopJPae+JKi5r4KayqqyM1IpTDCHMXGGNORYt199DhwDPAjYCZQAtwLbAce9TuwZIk4xEV6rqc5fztSj6x0euaks6aiirVehsw2xpgOEKv6aIyqjnXnTihV1cPd9f8WkQUJiC0pNlYGyExLoSA4C1QcHdc6WklhDmvcNoWhRblt72CMMbsoVkmhFkBV63HuOArVesLc3cSGyhr6F2Q1fyv3OO6RHwa7ScH6KBhjEiVWSWGQiNwNSMhz3OWBvkeWJBu2VTc3MgPsLEtqSeH1hU5fhZJ2DPdsjDHxipUUrg95PifstfDl3caGygATSno1r6gqh6KRSYkltHRgfRSMMYkQKykMBKar6meJCibZVJWNlTXNjczQai6FRApNClZ9ZIxJhFhJYSVwlYjsBywApgNvquqWhESWBFuq6qitb2yuPqqvgdrtSU8K3X3IbGNM4sQaOvsZ4BkAERkPTAFeFJFU4D/Av1X1vwmJMkGabkctCLkdFZLW0Ny/IIvUFKFPfmbzMN7GGOMjTwPSu1VInwF/FJEewNHARcDulRQqnbl9W3dcS05JIS01hYE9s1tWZxljjI/aTAoi0geYDAwAqoHPgX+p6gs+x5ZwG7bVAGHTcELSkgLAz6fs2a4pPY0xpj1izdF8JHADUIhTStiEM8nOKcAIEXkeuFNVKxMRaCJsqAwgAr3zQ6bhBMhN3qgeJ+w7IGnnNsZ0P7FKCscDP1bVNeEvuL2cT8CpRtptSgwbtwUozsskPdXt09cJSgrGGJNIsRqar4/xWj3wki8RJZEz41rY7agA2b0i72CMMbsZL20KE4Fv0bJN4T+qWuFh3ynAXUAq8KCq3hb2+hDgYaA3UAGcq6ql8b6JjrKxMtCyk1hVOWQVQKrV6RtjuodYQ2efLyLzgBuBbGAZTrvCocBbIvKYiJTE2D8VuA84DhgDnCUiY8I2+zPwuKruC9wC/HFX3syuWr8tvKSQvMHwjDEmGWKVFHKByapaHelFERkHjAJatTm4DgRWqOpKd/tngJOBxSHbjKF5Frd3SWKVVKCugW3Vdc23o4IlBWNMtxO1pKCq90VLCO7r81X17RjHHgisDVkupfVAeguA09zn3wPyRaTVp7CIXCwic0RkzubNm2Ocsv1azbgGlhSMMd1OrFtS7472GoCqXtnGsSPNCKNhy9cB94rI+cD7wDqgPsK5pgJTASZOnBh+jA7RNONaeFLot48fpzPGmE4pVvXR3F08dikwOGR5EGHzMqjqN8CpACKSB5ymqtt28bztsjF8Gk7VpM6lYIwxyRDrltTHQpdFJFdVd8Zx7NnAKBEZhlMCOBM4O+yYxUCFqjbiNGg/HMfxO9T68HGP6qqgPmDVR8aYbiXWzGsAiMgkEVkMLHGX9xORv7W1n9uX4QpghrvvNFX9QkRuEZGT3M2OAJaJyJdAX+DW9r2NXbdhW4C8zDTyMt08aR3XjDHdkJcB8f4POBZ4BUBVF4jIYV4OrqpvAG+Erbsp5PnzwPOeo/XRxsoAfXtkNq+wpGCM6YbaLCkAqOrasFW73RzNGyoDrW9HBUsKxphuxUtSWCsihwAqIhkich1uVdLuor6hkRWbdjCkKLd5ZdNcCpYUjDHdh5ekcClwOU4fg3XAOHd5t7Fo3Ta2B+o5ZERIArCSgjGmG2qzTUFVy4BzEhBL0ny4ogyAQ0aEDJFdVQ6SAlk9kxSVMcYknpe7j4aLyKsisllENonIyyIyPBHBJcqsFWXsPaBUO86JAAAgAElEQVQHhbkZzSuryiG7EFI8NbsYY8xuwcsn3lPANKA/zkipzwFP+xlUIlXV1jPv660cOjJsIh0b4sIY0w15SQqiqk+oar37eJLWw1V0WbNXb6G2oZHJrZJChSUFY0y3E2vso+D4Du+KyA3AMzjJ4Azg9QTElhCzlm8mIzWFA4aGDWdRVQ6Fu1UtmTHGtKmtsY+U5oHtLgl5TYHf+RVUIs1aUc7+Q3qRnZHa8oWqchh0QHKCMsaYJIk19tGwRAaSDGU7aliyvpLrj92z5QtNg+FZ9ZExpnvxMswFIjIWZ0Kcpi6/qvq4X0ElykdfOX0RWrUnBLZBY70lBWNMt+Nljubf4AxcNwZnHKPjgFlAl08KHy4vIz8rjX0GFrR8wTquGWO6KS93H50OHAVsUNULgP2AzNi7dH6qyqwVZRwyoojUlLD5gGyIC2NMN+UlKVS78x3Ui0gPYBPQ5W/L+bq8inVbq1v3TwArKRhjui0vbQpzRKQn8ADOHUk7gP/6GlUCzHKHtmjVngAhScFmXTPGdC9exj76ifv0HyLyb6AHUOZrVAkw7+st9MnPZFhxbusXg0khN0LCMMaY3Zinu4+CVHU1gIisAUr8CChRKgP1FOVlIiKtX6wqh9QMyMhLfGDGGJNE7R3tLcInaddSU99AVnqUtx/soxApYRhjzG6svUmhy499FKhrICstNfKLNu6RMaabijX20T1E/vAXoMtPMhCoa6Q4L8rbryq3RmZjTLcUq01hTjtf6xICdQ1kpUcrKZRDv7GJDcgYYzqBWGMfPRa+TkT6qeoGf0NKjEB9G0nBqo+MMd1QvG0Kb/gSRRIE6hojNzQ3NkD1FksKxphuKd6ksNvcjhOoayAzUkNz9VZALSkYY7qleJPCA75EkQQ1dY2Rq49siAtjTDfmZea1UM8E16tqhW9R+ayhUaltiFJ9VOV21ra7j4wx3ZDXmddKgC003466Buiyk/AE6hoArKRgjDFholYfqeowVR0OzABOVNViVS0CTgBe9HJwEZkiIstEZIU7z3P46yUi8q6IfCYiC0Xk+Pa+kXg0JYW0SCUFSwrGmO7LS5vCAaradNeRqk4HDm9rJxFJBe7DmZRnDHCWiIwJ2+xXwDRVHQ+cCfzNa+C7IlDfCNB6XmawpGCM6da8JIUyEfmViAwVkSEi8kug3MN+BwIrVHWlqtYCzwAnh22jOKOuAhQA33gNfFfErj6qgPRcSM9ORCjGGNOpeEkKZwG9gX+5j97uurYMBNaGLJe660LdDJwrIqU4fSB+GulAInKxiMwRkTmbN2/2cOrYgkkh4i2p1nHNGNONeZlPoQK4SkTyVHVHHMeO1KchfCyls4BHVfVOEZkEPCEiY92Z3kJjmApMBZg4ceIuD8YXqHMOH/nuIxv3yBjTfbVZUhCRQ0RkMbDYXd5PRLzU/ZcCg0OWB9G6euhCYBqAqn4MZAG+z2xT09bdR1ZSMMZ0U16qj/4KHIvbjqCqC4DDPOw3GxglIsNEJAOnIfmVsG3WAEcBiMhonKSw6/VDbQjUW1IwxphIPPVoVtW1YasaPOxTD1yBc0vrEpy7jL4QkVtE5CR3s2uBH4vIAuBp4HxV9X2uhtjVRzaXgjGm+/IyHedaETkEUPcb/5U4H/Jtcm9lfSNs3U0hzxcDk72H2zGa+ymElRTqa6Gm0pKCMabb8lJSuBS4HOfOoVJgHPATP4PyW3NJISwpVLsjd1hDszGmm/JSUthTVc8JXSEik4EP/QnJf839FMJyonVcM8Z0c15KCvd4XNdlRG1otqRgjOnmYo2SOgk4BOgtIteEvNQDiDJlWdcQrD7KDB/7yJKCMaabi1V9lAHkudvkh6yvBE73Myi/ORPspCAS1r9uZ3DYbEsKxpjuKdYcze8B74nIo6r6dQJj8l2gLsr8zFXW0GyM6d68tCk8KCI9gwsi0ktEZvgYk++cpBBliIusAkhNT3xQxhjTCXhJCsWqujW4oKpbgD7+heS/QF0j2dab2RhjWvGSFBpFpCS4ICJDaD2wXZcSvfrIkoIxpnvz0k/hl8AsEXnPXT4MuNi/kPwXqG8kM1pS6DEg8QEZY0wn4WXo7H+LyATgYJzhsK9W1TLfI/NRoK4hylScFdBvn8QHZIwxnUTU6iMR2cv9OQEowRn2eh1Q4q7rsmpiVh/ZnUfGmO4rVknhWuDHwJ0RXlPg275ElACBusbWdx/VVkF9tbUpGGO6tVj9FH7s/jwyceEkRqA+QknBejMbY0zMYS5OjbWjqr7Y8eEkhtOmYEnBGGPCxao+OtH92QdnDKR33OUjgZlAF04KEaqPLCkYY0zM6qMLAETkNWCMqq53l/sD9yUmPH9E7KfQNMSFJQVjTPflpfPa0GBCcG0E9vApHt81Nio1kfopWEnBGGM8dV6b6Y519DTOXUdnAu/6GpWPauqjzM9cVQaSAlk9I+xljDHdg5fOa1eIyPdwejIDTFXVf/kbln+izs9cVQ7ZhZDipfBkjDG7Jy8lBYB5wHZV/Y+I5IhIvqpu9zMwv8Scdc2qjowx3VybX4tF5MfA88D97qqBwEt+BuWn4KxrrauPKiwpGGO6PS91JZcDk3FmXENVl9OFh85uqj6KWFKwIS6MMd2bl6RQo6q1wQURSaMLD50dTAqt5lOw6iNjjPGUFN4Tkf8FskXkaOA54FV/w/JPsPooM7T6SNWSgjHG4C0p3ABsBhYBlwBvAL/yMyg/RWxorqmExnpLCsaYbi/m3Ucikgo8pqrnAg8kJiR/1US6JdU6rhljDNBGSUFVG4DeIpLRnoOLyBQRWSYiK0Tkhgiv/1VE5ruPL0Vka6TjdKSIdx/ZEBfGGAN466ewGvhQRF4BdgZXqupfYu3kljLuA44GSoHZIvKKqi4OOcbVIdv/FBgfV/TtEPHuIyspGGMM4C0pfOM+UoD8OI59ILBCVVcCiMgzwMnA4ijbnwX8Jo7jt0vspGC3pBpjure22hR6A6/jfLjHW7UzEFgbslwKHBTlPEOAYTQPzx3++sXAxQAlJSVxhtFSINLYR1ZSMMYYIPYczRcBXwD3AEtF5KQ4jy0R1kXr33Am8LzbhtF6J9WpqjpRVSf27t07zjBaijj2UVU5pKRDZjwFIWOM2f3Eamj+GbC3qk7CmWTnxjiPXQoMDlkehFMNFcmZOKOw+i5Q10hGagopKSE5K9hHQSLlMWOM6T5iJYVaVd0M4LYLZMZ57NnAKBEZ5t69dCbwSvhGIrIn0Av4OM7jt0ugrqFlxzWAneWQW5yI0xtjTKcWq01hkIjcHW1ZVa+MdWBVrReRK4AZQCrwsKp+ISK3AHNUNZggzgKeUdWEDJ0RedY1G/fIGGMgdlK4Pmx5brwHV9U3cHpAh667KWz55niPuyucpBBhfuZ+YxMZhjHGdEqx5mh+LJGBJEqgrjHyBDt255ExxsS8+2iqiET8+iwiuSLyIxE5x7/Q/BGoD6s+amyA6i2WFIwxhtjVR38DbhKRfYDPcQbFywJGAT2Ah4F/+h5hBwvUNbQcNrt6K6CWFIwxhtjVR/OBH4hIHjAR6A9UA0tUdVmC4utwgbpG8rNC3rZ1XDPGmCZtDnOhqjuAmQAi0ouWfQ+6nEBdA73zQ+6utSEujDGmiZc5mmeKSA8RKQQWAI+IyF/9D80fNfWNNhieMcZE4WWSnQJVrQROBR5R1f2Bo/wNyz+Bugay0mzcI2OMicRLUkgTkf7AD4DXfI7Hd606rwWTQrZVHxljjJekcAtOr+SvVHW2iAwHlvsbln8CdY2tR0hNz4GMnOQFZYwxnYSXhubngOdCllcCp/kZlF9UtXU/haoKqzoyxhiXl4bm4SLyqohsFpFNIvKyiAxLRHAdrbahEdUIE+xYUjDGGMBb9dFTwDScfgoDcEoNz/gZlF+C8zNnhjc0W1IwxhjAW1IQVX1CVevdx5NEnyynU4s8FWeZJQVjjHFFbVNw+yUAvCsiN+CUDhQ4A2eKzi4nclKwNgVjjAmK1dA8FycJBKcjuyTkNQV+51dQfglWHzXdfVRfCzWVlhSMMcYVa+yjqI3JIpLuTzj+ajU/c3WF89OGuDDGGMBbmwIA4vi2iDyIM/9yl9Oq+sh6MxtjTAtebkk9SETuAr7GmWP5A2AvvwPzQ6DeqT7KznDftiUFY4xpIdYkO7eKyHLgD8AiYDywWVUfU9UtiQqwIwVLCplpVlIwxphIYjU0XwwsA/4OvKaqARHpkreiBln1kTHGxBar+qgfcCtwErBCRJ4AskWkzaExOqua8LuPqqyh2RhjQsW6+6gBmA5MF5Es4AQgB1gnIm+r6tkJirHDBOojlBQyCyC1S95MZYwxHc7Tt35VDQDPA8+LSA/ge75G5ZOI1UdWSjDGmCaxGprPFZFWr6tqpao+JiIjRORQf8PrWE2d19JC7j6y9gRjjGkSq6RQBHwmInNxejdvBrKAkcDhQBlwg+8RdqBAXQNpKUJaakhSyO+f3KCMMaYTidWmcJeI3At8G5gM7AtUA0uAH6rqmsSE2HGcCXbCxj3qOzZ5ARljTCcTs03BbWx+y33ETUSmAHcBqcCDqnpbhG1+ANyMM57SAj8bsJ0JdsKHzbY2BWOMCWqzodmdUOenwNDQ7VX1pDb2SwXuA47GGRZjtoi8oqqLQ7YZBdwITFbVLSLSpz1vwqtAbUNzx7XaKqirsjYFY4wJ4eXuo5eAh4BXgcY4jn0gsMKdvhMReQY4GVgcss2PgfuCPaRVdVMcx49bi5KCdVwzxphWvCSFgKre3Y5jDwTWhiyXAgeFbbMHgIh8iFPFdLOq/jv8QCJyMU4Pa0pKStoRiqNFm4IlBWOMacVLUrhLRH4DvAnUBFeq6rw29pMI68KHyUgDRgFHAIOAD0RkrKpubbGT6lRgKsDEiRPbPdRGoK7BkoIxxsTgJSnsA/wQ5y6kYPWRusuxlAKDQ5YHAd9E2OYTVa0DVonIMpwkMdtDXHEL1DWQnRFMCsEhLiwpGGNMkJek8D1guKrWxnns2cAot6F6HXAmEH5n0UvAWcCjIlKMU520Ms7zeBaoa6QwN8NZsJKCMca04mWSnQVAz3gPrKr1wBXADJy+DdNU9QsRuUVEgncuzQDKRWQx8C5wvaqWx3surwL1DWSGVh9JCmQV+HU6Y4zpcryUFPoCS0VkNi3bFGLekupu8wbwRti6m0KeK3CN+/BdTV1j81ScVeWQ3QtSUmPvZIwx3YiXpPAb36NIEKeh2cY9MsaYaNpMCqr6XiICSYRWdx9ZUjDGmBa89GjeTvOtpBlAOrBTVXv4GZgfAvWNLSfYKRyW3ICMMaaT8VJSyA9dFpFTcHordyl1DY00NGrLNoVBE5MblDHGdDJe7j5qQVVfou0+Cp1Oiwl2VK36yBhjIvBSfXRqyGIKMJHWPZM7vUDo/Mw126GxzpKCMcaE8XL30Ykhz+uB1TgD23UpwZJCZnoqVJU5Ky0pGGNMC17aFC5IRCB+a1F9VLXRWWlJwRhjWmizTUFE7hCRHiKSLiJvi0iZiJybiOA6Uov5mW2IC2OMichLQ/MxqloJnIAzgN0ewPW+RuWDQH1oSSGYFGzWNWOMCeUlKaS7P48HnlbVCh/j8U3L6iMrKRhjTCReGppfFZGlQDXwExHpDQT8Davjtbj7qKocUtIhM7+NvYwxpntps6SgqjcAk4CJ7rwHVXThu4+ygyWFnCKQSPMAGWNM9+Wp85qqblHVBhGZqqo7VXWD34F1tJbVRxVWdWSMMRHE26O5y44LEah3qo8yg9VH1shsjDGtxJsUNvkSRQLUhDc0W0nBGGNaiSspqOoUvwLxW7+CLL41qtgZEK+qHHKLkx2SMcZ0OnEPiAcgIlM7OhC/nbDvAJ648CAyUhSqt1hJwRhjIoh6S6qIRKt0F5w+C11TYBtooyUFY4yJIFY/hc3A1zhJIEjd5T5+BuUr67hmjDFRxUoKK4GjVHVN+Asista/kHxmQ1wYY0xUsZLC/wG9gFZJAbjDn3ASYKcNm21Md1FXV0dpaSmBQJcbhKHdsrKyGDRoEOnp6W1vHEHUpKCq98V47Z52na0zsOojY7qN0tJS8vPzGTp0KNINRjBQVcrLyyktLWXYsPbNQd+uu4+6tGBSyLbqI2N2d4FAgKKiom6REABEhKKiol0qGXXPpJCeAxk5yY7EGJMA3SUhBO3q++2GScHGPTLGmGi6YVKwcY+MMYlRXl7OuHHjGDduHP369WPgwIFNy7W1tZ6OccEFF7Bs2TKfI23mZT4FRGSeqk6IthxjvynAXUAq8KCq3hb2+vnAn4B17qp7VfVBj7G3j417ZIxJkKKiIubPnw/AzTffTF5eHtddd12LbVQVVSUlJfJ39EceecT3OEN5SgrhCcBjQkgF7gOOxpnGc7aIvKKqi8M2fVZVr/AY766rKofC9rXKG2O6rt+++gWLv6ns0GOOGdCD35y4d9z7rVixglNOOYVDDz2UTz/9lNdee43f/va3zJs3j+rqas444wxuuukmAA499FDuvfdexo4dS3FxMZdeeinTp08nJyeHl19+mT59OrYvcZvVRyJyhYj0bMexDwRWqOpKVa0FnqEzTM5TVQE5NhieMSa5Fi9ezIUXXshnn33GwIEDue2225gzZw4LFizgrbfeYvHi8O/PsG3bNg4//HAWLFjApEmTePjhhzs8Li8lhX7AHBGZBzwMzFBV9bDfQCC053MpcFCE7U4TkcOAL4GrVbVVb2kRuRi4GKCkpMTDqaNoqIOabVZ9ZEw31J5v9H4aMWIEBxxwQNPy008/zUMPPUR9fT3ffPMNixcvZsyYMS32yc7O5rjjjgNg//3354MPPujwuLxMx/krYBTwEHA+sFxE/iAiI9rYNdJ9UeHJ5FVgqKruC/wHeCxKDFNVdaKqTuzdu3dbIUdXVeH8tIZmY0yS5ebmNj1fvnw5d911F++88w4LFy5kypQpEfsaZGRkND1PTU2lvr6+w+PyOh2nAhvcRz3O8BfPi0is4S5KgcEhy4OAb8KOW66qNe7iA8D+HuNuH+vNbIzphCorK8nPz6dHjx6sX7+eGTNmJC2WNquPRORK4DygDHgQuF5V60QkBVgO/DzKrrOBUSIyDOfuojOBs8OO3V9V17uLJwFL2vUuvLKkYIzphCZMmMCYMWMYO3Ysw4cPZ/LkyUmLRdpqHhCRW4CHVPXrCK+NVtWoH+QicjzOwHqpwMOqeqt7vDmq+oqI/BEnGdQDFcBlqro0VjwTJ07UOXPmtPW+IvviJXjuPLjsY+g7pu3tjTFd2pIlSxg9enSyw0i4SO9bROaq6sS29vXS0PwGzgd28MD5wBhV/TRWQgBQ1Tfc/UPX3RTy/EbgRg8xdAwrKRhjTExe2hT+DuwIWd7prut6bC4FY4yJyUtSkNBbUFW1EY+d3jqdqnLILIDU9o0zbowxuzsvSWGliFwpIunu4yqcWdm6Hhv3yBhjYvKSFC4FDsG5gyjYAe1iP4PyjY17ZIwxMbVZDaSqm3BuJ+36qsohr1+yozDGmE7Ly9hHWSJyuYj8TUQeDj4SEVyHs7kUjDEJlpqa2jRc9rhx47jtttva3imCI444gnbfjh8HLw3GTwBLgWOBW4Bz8LuTmV+sTcEYk2DZ2dlNw2d3BV6SwkhV/b6InKyqj4nIU0Dy+mC3V20V1FVBro2Qaky3NP0G2LCoY4/Zbx84Lv5v/tOnT+eRRx5h2rRpAMycOZM777yTV199lcsuu4zZs2dTXV3N6aefzm9/+9uOjbkNXpJCnftzq4iMxRn/aKhvEfmlOjgYnlUfGWMSp7q6mnHjxjUt33jjjZx22mlccskl7Ny5k9zcXJ599lnOOOMMAG699VYKCwtpaGjgqKOOYuHChey7774Ji9dLUpgqIr2AXwGvAHnAr32Nyg/Wm9mY7q0d3+g7QrTqoylTpvDqq69y+umn8/rrr3PHHc74otOmTWPq1KnU19ezfv16Fi9e3HmSgjvoXaWqbgHeB4YnJCo/WFIwxnQiZ5xxBvfddx+FhYUccMAB5Ofns2rVKv785z8ze/ZsevXqxfnnnx9xCG0/xbz7yO29nLipMv1UZdVHxpjO44gjjmDevHk88MADTVVHlZWV5ObmUlBQwMaNG5k+fXrC4/JSffSWiFwHPIsz7hEAqloRfZdOyEoKxpgkCG9TmDJlCrfddhupqamccMIJPProozz2mDO/2H777cf48ePZe++9kzaEtpehs1dFWK2qmpSqpHYPnb30dZj/FPzgcUhJ7fjAjDGdjg2d3azDhs5W1WG7EFvnsdd3nYcxxpiovMy89j+R1qvq4x0fjjHGmGTy0qZwQMjzLOAoYB5gScEY0+mpKiKS7DASpq0mgbZ4qT76aeiyiBTgDH1hjDGdWlZWFuXl5RQVFXWLxKCqlJeXk5WV1e5jtGeynCpgVLvPaIwxCTJo0CBKS0vZvHlzskNJmKysLAYNGtTu/b20KbwKBMsjKcAYYFq7z2iMMQmSnp7OsGG7x70yieKlpPDnkOf1wNeqWupTPMYYY5LIS1JYA6xX1QCAiGSLyFBVXe1rZMYYYxLOy3SczwGNIcsN7jpjjDG7GS8lhTRVrQ0uqGqtiGT4GFNMc+fOLRORr9u5ezFQ1pHxdBCLKz4WV/w6a2wWV3x2Ja4hXjbykhQ2i8hJqvoKgIicvAtB7TJV7d3efUVkjpdu3olmccXH4opfZ43N4opPIuLykhQuBf4pIve6y6VAxF7OxhhjujYvnde+Ag4WkTycAfS2+x+WMcaYZGizoVlE/iAiPVV1h6puF5FeIvL7RATng6nJDiAKiys+Flf8OmtsFld8fI/Ly9DZn6nq+LB181R1gq+RGWOMSTgvt6SmikhmcEFEsoHMGNsbY4zporw0ND8JvC0ij+AMd/EjbIRUY4zZLbVZUlDVO4DfA6OBvYHfqertfgfW0URkiogsE5EVInJDgs89WETeFZElIvKFiFzlrr9ZRNaJyHz3cXzIPje6sS4TkWN9jG21iCxyzz/HXVcoIm+JyHL3Zy93vYjI3W5cC0XElypEEdkz5JrMF5FKEflZMq6XiDwsIptE5POQdXFfHxE5z91+uYic51NcfxKRpe65/yUiPd31Q0WkOuS6/SNkn/3d3/8KN/ZdGko0Slxx/946+v81SlzPhsS0WkTmu+sTeb2ifTYk729MVeN6AJOB++LdL5kPIBX4ChgOZAALgDEJPH9/YIL7PB/4EmdgwZuB6yJsP8aNMRMY5sae6lNsq4HisHV3ADe4z28AbnefHw9MBwQ4GPg0Qb+7DTgdbxJ+vYDDgAnA5+29PkAhsNL92ct93suHuI7B6WwKcHtIXENDtws7zn+BSW7M04HjfIgrrt+bH/+vkeIKe/1O4KYkXK9onw1J+xvz0qaAiIwTkdtFZDVOqWGpl/06kQOBFaq6Up3e2c8AJyfq5Kq6XlXnuc+3A0uAgTF2ORl4RlVrVHUVsALnPSTKycBj7vPHgFNC1j+ujk+AniLS3+dYjgK+UtVYvdh9u16q+j5QEeF88VyfY4G3VLVCVbcAbwFTOjouVX1TVevdxU+AmOMnu7H1UNWP1flkeTzkvXRYXDFE+711+P9rrLjcb/s/AJ6OdQyfrle0z4ak/Y1FTQoisoeI3CQiS4B7cTqtiaoeqar3tOdkSTQQWBuyXErsD2XfiMhQYDzwqbvqCrcY+HCwiEhi41XgTRGZKyIXu+v6qup6cP5ogT5JiCvoTFr+syb7ekH81ycZ1+1HON8og4aJyGci8p6IfMtdN9CNJRFxxfN7S/T1+hawUVWXh6xL+PUK+2xI2t9YrJLCUpxvaSeq6qFuImhoz0k6gUj1frs2Z117gnA6AL4A/ExVK4G/AyOAccB6nCIsJDbeyercXnwccLmIHBZj24ReR3HG2DqJ5gEYO8P1iiVaHIm+br/EGeb+n+6q9UCJOreWXwM8JSI9EhhXvL+3RP8+z6LlF4+EX68Inw1RN40SQ4fFFispnIZTl/uuiDwgIkdFOXFXUAoMDlkeBHyTyABEJB3nl/5PVX0RQFU3qmqDqjYCD9Bc5ZGweFX1G/fnJuBfbgwbg9VC7s9NiY7LdRwwT1U3ujEm/Xq54r0+CYvPbWA8ATjHreLArZ4pd5/Pxamv38ONK7SKyZe42vF7S+T1SgNOBZ4NiTeh1yvSZwPJ/Bvz0BCSC5wDvIYzFeffgWN2pXEl0Q+cW29X4jRmBRuu9k7g+QWn/vH/wtb3D3l+NU79Kjh3eYU2wK3Eh4Zm93ebH/L8I5x6yD/RspHrDvf5d2nZyPVfn6/bM8AFyb5ehDU8xnt9cBr/VuE0APZynxf6ENcUYDHQO2y73sHrgdN4uy54fmC2G2uw4fR4H+KK6/fm1/9reFwh1+y9ZF0von82JO1vLN43UAhcAryzq7+gRD9wWu2/xMn6v0zwuQ/FKcotBOa7j+OBJ4BF7vpXwv55funGuoxdvMMhRlzD3X+4BcAXwesCFAFvA8vdn8F/CAHuc+NaBEz08ZrlAOVAQci6hF8vnGqF9UAdzrexC9tzfXDq+Fe4jwt8imsFTr1y8G/sH+62p7m/3wXAPJwq4eBxJgKfuzHfizvKQQfHFffvraP/XyPF5a5/FLg0bNtEXq9onw1J+xtrc5gLY4wx3YenW1KNMcZ0D5YUjDHGNLGkYIwxpoklBWOMMU0sKRhjjGliScHsFkSkKGRUyw1ho3JmeDzGIyKyZxvbXC4i53RQzCe78S0QkcUicpG7/lQR2asjzmFMvOyWVLPbEZGbgR2q+uew9YLzN9+YlMBaxpKJ08Fooqp+4y4PUdUvReRJ4HlVfSm5UZruyEoKZrcmIiNF5HN3TPx5QH8RmSoic9zx628K2XaWOyJwmohsFZHb3G/xH4tIH3eb34vIz0K2v01E/ivO2P+HuOtzReQFd9+n3XONCwutAKcjUgU0Da3wpTv42vHAX1OLowIAAAIlSURBVN1SxFARGSUiM9xBC98XkT3c8zwpIn8XkQ9E5EsROc7ny2m6AUsKpjsYAzykquNVdR3O8AETgf2Ao0VkTIR9CnCGP9gP+Bint2gkoqoHAtcDwQTzU2CDu+9tOCNftqDOWFMzgK9F5CkROUtEUlT1A+AN4GpVHaeqq3Ema/+Jqu4P3IjTkzZoMHA4cCIwVUKmzjWmPSwpmO7gK1WdHbJ8lojMwyk5jMZJGuGqVTU49PRcnHFzInkxwjaH4ozbhKoGhxBpRVXPB44G5uCMbzM1fBtxZk87GHhBnJnB7gMGhGwyTVUbVXUZzhAXo6LEaYwnXuZoNqar2xl8IiKjgKuAA1V1q1t/nxVhn9qQ5w1E/1+pibCN59GEVXUhsFBEnsKZYOWisE0EKFPV8OqnpkO0sWxMXKykYLqbHsB2oDJkxqqONgtnJi9EZB8ilEREpEfY3BXjgODscttxpmZEnVm01ovI99z9UkRkv5D9vi+OPXCqkkInijEmblZSMN3NPJzhpT/HGZ75Qx/OcQ/wuIgsdM/3ObAtbBsBbhSRB4BqYAfN7RZPA/eLyLU40zCeCfzdvasqA3gSZwRPcEbEfB9nZq6L1Zm+0ph2s1tSjelg7sQtaaoacKur3gRGafP8yR11Hrt11XQ4KykY0/HygLfd5CDAJR2dEIzxi5UUjDHGNLGGZmOMMU0sKRhjjGliScEYY0wTSwrGGGOaWFIwxhjT5P8Bjj8aZA87f5AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title('Accuracy versus training step.')\n",
    "plt.xlabel('Training Step')\n",
    "plt.ylabel('Accuracy: 1-sum(Abs(Predicted-Label))/Batchsize')\n",
    "plt.plot(tr[:,0],tr[:,2],label='Train')\n",
    "plt.plot(ev[:,0],ev[:,2],label='Eval')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exporting/Saving the model to file\n",
    "<p>The standard output format of TensorFlow is the SavedModel format containing a protobuf (*.pb) file and a variables directory containing the values of the trainable variables.  Flogo reads this in assuming that the protobuf file contains the appropriate metadata.  Below we use the simple_save function to define the metadata (containing inputs and outputs) and write to file.  We also use the write_graph function to export the protobuf into readable text.  However this *pbtxt file does not contain the metadata.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "154031266409\n",
      "INFO:tensorflow:Assets added to graph.\n",
      "INFO:tensorflow:No assets to write.\n",
      "INFO:tensorflow:SavedModel written to: ./SimpleCNNModel/154031266409/saved_model.pb\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./SimpleCNNTextModel/154031266409/train.pbtxt'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t=\"%d\"%int(time.time()*100)\n",
    "print(t)\n",
    "path=\"./SimpleCNNModel/{:s}/\".format(t)\n",
    "inputs={\"X\":X}\n",
    "outputs={\"pred\":pred}\n",
    "tf.saved_model.simple_save(sess, path,inputs=inputs,outputs=outputs)\n",
    "tf.train.write_graph(sess.graph, './SimpleCNNTextModel/{:s}/'.format(t), 'train.pbtxt',as_text=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
